1.Introduction
The telecommunication in Korea has shown rapid growth under the fierce competition among the big 3 operators during the last decade. It is important to note that study on performance evaluation of the telecommunication branches has been a neglected issue.
The objectives of this work are threefold. First, we evaluate the efficiency of the telecommunication in Korea, specifically in the branch level, taking the environmental factor into account. Second, we examine the priority of key performance indicators to enable a suitable performance evaluation. Lastly, we provide individualized operating benchmark targets for performance improvement. This is a necessary guide to optimize operation with limited resources.
We use the data envelopment analysis to evaluate the performance of the telecommunication branch. The methodology was implemented in four steps. First, DEA superefficiency was analyzed. With the analysis, outliers were identified and the efficient units were ranked in order [1]; second, using the extended CCR model developed by Banker and Morey [3], we consider the environmental (nondiscretionary) factor that affects the performance; and finally, a sensitivity analysis was carried out by deleting variables from the base model to determine the contribution of each variable on the efficiency.
This is one of the few studies that evaluate the performance of telecommunication branches. We propose the method in which the key variables were selected on branch efficiency using DEA in conjunction with the information provided by nondiscretionary factor, superefficiency model, and the sensitivity analysis.
2.Literature Review
Assessing the efficiency of the organization has been studied in many ways. Although efficiency in the organization has also been analyzed using an econometric approach, the most widely used method has been frontier methods such as Data Envelopment Analysis [11, 21, 22].
In order to evaluate performance in the telecom branches, input and output indicators should be selected carefully because DEA results are sensitive to the selection of variables [4]. Hence, we reviewed all of the available studies to determine the input/output variables to be used in our analysis. The empirical works in telecom industry using DEA are shown in <Table 1>.
These studies consider the efficiency of telecom operators from a broad perspective, yet only Cooper et al. [7] has studied the efficiency of branch level in a Korean mobile telecommunication company. Our study has a more specific purpose than previous studies, as it attempts to evaluate branch performance in conjunction with KPIs (key performance indicators) considering nondiscretionary factor. We can claim the importance of including the environmental variables in performance analysis, as factors like population and competition status have a clear impact on the performance of branches [5, 13].
3.Method and Data
3.1.Method
This work adopts the inputoriented DEA model. Since the telecom branches are homogeneous groups, no differential factor could cause any of them to have an advantage over the others. We assumed constant returns to scale existed. The linear program used to obtain the level of efficiency of each DMU was :
$\begin{array}{l}\text{Min}\hspace{0.17em}{\theta}_{\text{k}}\in \left(\Sigma {\text{S}}_{\text{i}}{}^{\text{}}+\Sigma {\text{S}}_{\text{r}}{}^{+}\right)\\ \text{s.t.}\\ {\theta}_{\text{k}}\hspace{0.17em}{\text{X}}_{\text{ik}}\hspace{0.17em}=\hspace{0.17em}\Sigma {\text{X}}_{\text{ij}}{\lambda}_{\text{j}}+{\text{S}}_{\text{i}}{}^{\text{}}\hspace{1em}\hspace{1em}\hspace{1em}\left(\text{i}\hspace{0.17em}=\hspace{0.17em}1,\hspace{0.17em}2,\hspace{0.17em}\cdots ,\hspace{0.17em}\text{m}\right)\\ {\text{Y}}_{\text{rk}}=\hspace{0.17em}\Sigma {\text{Y}}_{\text{rj}}{\lambda}_{\text{j}}{\text{S}}_{\text{r}}{}^{\text{+}}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{0.17em}\left(\text{r}\hspace{0.17em}=\hspace{0.17em}1,\hspace{0.17em}2,\hspace{0.17em}\cdots ,\hspace{0.17em}\text{s}\right)\\ \lambda \text{j,}\hspace{0.17em}{\text{S}}_{\text{i}}{}^{\text{}}{\text{,S}}_{\text{r}}{}^{\text{+}}\ge \in >0\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{0.17em}\left(\text{j}\hspace{0.17em}\text{=}\hspace{0.17em}\text{1,}\hspace{0.17em}\text{2,}\hspace{0.17em}\cdots ,\hspace{0.17em}\text{n}\right)\end{array}$
Where ${\theta}_{\text{k}}$ is the parameter that measures the efficiency of the unit k (k = 1, ⋯, n); n is the total number of DMU. Y_{rj} is the amount of output r generated by unit j; X_{ij} is the amount of input i used by unit j; λj is weight. ${\text{S}}_{\text{i}}{}^{}$ is slack variable for input; ${\text{S}}_{\text{r}}{}^{\text{+}}$ is slack variable for output; $\in $ is a small positive number. It is possible to improve efficiency by the total slack values for each input and output.
Second, we use the extended CCR model [6] to investigate the influence of nondiscretionary (ND) factor. The original DEA assumed that all variables are discretionary, that is, can be managed at the discretion of managers. However, the variables that are beyond the control of management may influence the level of efficiency. The modification to incorporate ND factors is given by Banker and Morey [3]
$\begin{array}{l}\text{Min}\hspace{0.17em}{\theta}_{\text{k}}\in \left(\Sigma {\text{S}}_{\text{i}}{}^{\text{}}+\Sigma {\text{S}}_{\text{r}}{}^{+}\right)\\ \text{s.t.}\\ {\theta}_{\text{k}}\hspace{0.17em}{\text{X}}_{\text{ik}}\hspace{0.17em}=\hspace{0.17em}\Sigma {\text{X}}_{\text{ij}}{\lambda}_{\text{j}}+{\text{S}}_{\text{i}}{}^{\text{}}\hspace{1em}\hspace{1em}\hspace{1em}\left(\text{i}\hspace{0.17em}\in {\text{I}}_{\text{D}}\right)\\ {\text{X}}_{\text{ik}}=\hspace{0.17em}\Sigma {\text{X}}_{\text{ij}}{\lambda}_{\text{j}}{\text{+S}}_{\text{i}}{}^{\text{}}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{0.17em}\left(\text{i}\hspace{0.17em}\in \hspace{0.17em}{\text{I}}_{\text{ND}}\right)\\ {\text{Y}}_{\text{rk}}\hspace{0.17em}=\hspace{0.17em}\Sigma {\text{Y}}_{\text{rj}}{\lambda}_{\text{j}}{\text{S}}_{\text{r}}{}^{\text{+}}\hspace{1em}\hspace{1em}\hspace{0.17em}\hspace{1em}\hspace{1em}\hspace{0.17em}\left(\text{r}\hspace{0.17em}\text{=}\hspace{0.17em}\text{1,}\hspace{0.17em}\text{2,}\hspace{0.17em}\cdots ,\hspace{0.17em}\text{s}\right)\\ \lambda \text{j}\hspace{0.17em}\ge \hspace{0.17em}\text{0}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{0.17em}\hspace{1em}\left(\text{j}\hspace{0.17em}\text{=}\hspace{0.17em}\text{1,}\hspace{0.17em}\text{2,}\hspace{0.17em}\cdots ,\hspace{0.17em}\text{n}\right)\end{array}$
Where ID, IND refer to discretionary (D) and nondiscretionary (ND) input I.
I = {1, 2, ⋯, m} = I_{D} ∪ I_{ND} with I_{D} ∩ I_{ND} = ϕ; ϕ is the empty set. The θ is minimized in the constraints for which i ∈ I_{D}, whereas the constraints for which i ∈ IND operate indirectly.
Third, we applied the superefficiency procedure for outlier identification and ranking of efficient units. Outliers which may introduce bias must be dealt with [1, 2].
Lastly, the sensitivity analysis was performed to define the priority of the KPIs. It allows the analyst to perform “whatif” scenarios on the DEA [17].
3.2.Data
Learned from the literature reviews, the input indicators are usually set labors, capital and facility, and the output indicators are set revenue, the number of subscribers. We also used the population of the telecom branch as a nondiscretionary factor. Population variable decides the consumer ability of a region that affects the revenue of a branch. Variables used in this study are as follows :

Discretionary input variables : Labor, Operating cost, Access lines

Discretionary output variables : Revenue, subscribers

Nondiscretionary input variable : Population of the regional branch
In <Table 2>, the Pearson correlation showed that the inputs will have a direct relation to the outputs. Test results comply with the principle of “isotonicity.”
Meanwhile, larger business units may form a cluster, showing that efficiency of scale is a major factor [19]. To test the effect of size, we divided our sample into quartiles by population, revenues and subscribers respectively. Then we compared the relative efficiencies of DMUs in the lowest quartile with the ones in the highest quartile. We found that the differences in relative efficiencies insignificant for each of the quartiles.
The case is a sample of 51 branches of a telecommunication operator in Korea (after this simply “the Telecom”). The telecom branch names were deleted and were named F1 to F51 DMU. The descriptive statistics for the data are as shown in <Table 3>.
4.Results
All DEA results were obtained by using the Efficiency Measurement System (EMS) developed by Holger Scheel [18]. Branches with superefficiency scores higher than 200% are regarded as an outlier. There is no branch with superefficiency score higher than 200%.
4.1.Influence of the NonDiscretionary Indicator
The final column of <Table 4> shows the influence of the nondiscretionary (ND) indicator on the efficiency level.
This result was obtained by comparing the results of efficiency with ND taken into account against the result with ND factor excluded. Therefore, they reveal how the population of the telecom branch influenced the efficiency results. In case of F11 and F47, these effects were 4.8% and 8.7% respectively. ND factor plays a meaningful role explaining the performance of branches.
4.2.Efficiency Analysis of Telecom Branch Performance
In an inputoriented superefficiency model, all scores may be either greater than the unit (superefficient), equal to the unit (efficient), or lower than the unit (inefficient). In the case of inefficiency, the gap with the unit indicates the level of inefficiency. For example, the efficiency score of F11 was 0.954, suggesting 4.6% inefficiency. This means that F11 would need to reduce its level of input by 4.6% to become efficient while leaving the output at its present value. However, this would not be the only action to this effect, as other additional measures will be required to achieve efficiency.
The values in the “potential for improvement” column of <Table 4> are obtained from the slack rate against each input and output. Those figures represent the required discretionary input reduction rate to achieve efficiency. Output wise, this explains how much more output should be added to reach efficiency level. That is, with respect to F11, apart from the input reduction of 4.6%, the revenue (O1) needs to be improved (O1 = 0.1%).
We can see the “virtual inputs and outputs”, i.e. the weights multiplied by the variables in <Table 4>. The optimal weights provide a measure of the relative contribution of variables to the overall efficiency. If we examine the virtual input and output, then we can see the relative influence of each variable. These values not only show, which variables contribute to the evaluation of DMU, but also to what extent they do so [8].
For example, in case of F11, total subscribers (O2, 95.4%) can explain its overall efficiency, which is influenced by the labor (I1, 52%), operating costs (I2, 28.3%) and access lines (I3, 19.7%).
The aggregated information from the last row “average” helps us to identify the indicators that should be improved and how each indicator contributes to the efficiency value. This implies that, generally speaking, the branches should focus more on reducing the level of access lines (I3 = 1.37%). Similarly, the results from the virtual values of the last row show that operating costs (I2 = 56.1%) and revenue (O1 = 52.8%) were key indicators. According to this analysis, efforts should be made to reduce the level of operating costs, meanwhile paying attention to the level of revenue.
4.3.Sensitivity Study of Evaluated Indicators
<Table 5> shows the overall efficiency based on different combination of indicators. The rates of change are negative or zero; the negative value means that the absence of the indicator will reduce DMU’s efficiency value. The average efficiency is decreased from 0.972 of the base case down to 0.863 in CASE2COST (deleting operating cost).
The labor gives the greatest influence to F13 (up to 17.3%) and minimum impact to F1, F34, F41, and F43. The sensitivity degree of indicators shows that F51 has always been efficient in all cases except for CASE3LINE. This confirms that F51 is more sensitive to access lines. On average, almost all DMUs are more sensitive to operating costs. The results suggest the priority as follows: operating costs > labor > subscribers > revenue > access line.
Using the sensitivity analysis, the branches could be categorized into 5 groups.

(1) RE : the DEA stays efficient or efficiency decreases very slightly. For example, F51 was robustly efficient.

(2) ME : the DEA is efficient in the base model and remains efficient in some situations, but efficiency decreases significantly in other situations. F1, F34 and F43 fall into this category.

(3) MI : the efficiency is above 0.9 but below 1 in the base model and stays in that range. F27 is a marginally inefficient branch.

(4) SI : the DEA efficiency is above 0.9 but below 1 in the base and drops to lower values. When labor is removed from the analysis, F11’s score changes to 0.822 (13.8%) which suggests that labor is the strength of F11.

(5) DI : the efficiency is significantly low (below 0.9) in all conditions. F48 can be considered as a distinctly inefficient.
Such distinction is useful for selecting branches for performance improvement. DI and SI branches clearly have problems and require attention. Since ME branches are very sensitive to changes in a few indicators, they need more attention than MI branches to prevent them from becoming inefficient. The efficiency of MI branches can be improved only based on a longtern plan because of their low sensitivity to changes in the indicators [17].
4.4.Case Study : Performance Optimization Report
Performance optimization report can be provided for each branch. <Table 6> shows an example. F48’s relative efficiency score is 0.829 (0.842 with nondiscretionary) and is classified as distinctively inefficient.

(1) Reference sets : if F48 hopes to improve its relative efficiency, it is suggested for F48 to refer to benchmarks (reference sets). That is, F48 is suggested to refer to F24, F51, F26 and F17 at 73.8%, 4.8%, 1.3%, and 1.0% respectively in setting the benchmark target for its input and output.

(2) Improvement : all indicators of F48 should be reduced to 17.05%to reach benchmark target. All of its outputs can be maintained at the same level. It has zero slack in outputs.

(3) Contribution : the input and output items contribute to F48’s relative efficiency. I2 (operating costs 51.0%) is the major input indicator and O1 (revenue 54.8%) is the major output indicator.

(4) Sensitivity degree : this shows the I2 (operating costs) has the most important impact on F48’s relative efficiency; deleting I2 can reduce F48’s relative efficiency down to 13.7%.
As a result, we conclude that the indicators’ priority for F48 is operating costs > revenue > subscribers > labor > access line. Even though labor (I1) comes fourth in priority, deleting I1 helps F48 to raise 2 steps in its ranking among all DMUs.
5.Conclusion
This paper examines the efficiency in the telecom branches by considering nondiscretionary factor and identifies which KPIs (key performance indicators) are important to the organization. Inefficient branches can improve their performance by checking the room for potential improvement (slack); they can also get ideas for performance improvement by benchmarking efficient branch from their reference set. In addition, sensitivity analysis helps the branch to learn the influence its inputs and outputs give to the performance. The results show that average efficient score decreases from 0.972 (base case) to 0.863 for CASE2COST. The average score of the data proves the priority of operating cost over other indicators.
This paper offers significant contributions compared to past studies. First, it included the effect of nondiscretionary indicator when developing the performance evaluation of the branch. The population effect was positive and improved overall efficiency by 0.91% on average. Second, using super efficiency approach, we tested the outliers and ranked the efficient DMUs. Third, the influence of each indicator was examined using information provided by the model (slacks, virtual input/output) and the sensitivity analysis of the KPIs. In addition, the example of the performance optimization report is presented as a guide for the managers to develop branch strategies. Managers can identify the topperforming units (reference sets), study best practices and adopt the strategy to the organization.
Despite the contributions, this paper has a few shortcomings and therefore further studies should be carried out. First, we use the Banker and Morey approach that includes nondiscretionary factors, but it is only applied to an input oriented model. However, there are theoretical alternatives that may be considered for future studies [9, 15]. Second, a dynamic model such as DEAwindows analysis was not used. Thus, this study cannot be generalized. Lastly, we did not consider the managerial difference. All other things being equal, management level is likely to influence the organization’s overall performance [20].
The organization’s performance may be evaluated with multiple tools. This paper addresses an issue for managers interested in evaluating the organization and improving efficiency. This study provides a valuable reference for application to future studies, exploring an analysis of KPIs. It will be interesting to examine the results of DEA in conjunction with the results of other measurement models.