Robert B. Cohen, Senior Fellow, Economic Strategy Institute, June 18, 2018

Challenge:

We examine the shift to AI and ML as part of an increasing sophistication in the analysis of processes and operations. We focus on this area since we believe that software plays a central role in shaping the direction and size of the economic benefits associated with how Cloud computing, AI and ML affect businesses. Hardware innovations are certainly part of the changes we expect to quantify.

Our goal is to estimate the magnitude and connection between software and hardware innovations and interactions between different types of software in the many areas that affect the way firms are using cloud computing, AI and ML. We believe if we can successfully develop this estimate, it will be possible to compare it to estimates of the impact of robotics and automation on the economy in terms of output and jobs.

There are several unique challenges to quantify the benefits that we want to measure:

1. There are many different factors shaping the dynamics within each area – we divide these into software development, infrastructure changes, and data analysis -- we want to quantify. Within each area, identifying what factor is the prime one is difficult, as is setting out the relationship between the factors driving the pace of change within a specific area.
2. Factors that change the efficiency of operations in one area, such as software development, can also impact the efficiency with which firms operate in another area, such as infrastructure. This means that factors not only have an impact in one of the areas driving economic benefits but may also impact the economic changes of another area. This is true of data we have for the use of containers:
   1. Containers (and very likely, Pods) speed up the rate at which developers can complete the build, test and deploy cycle for software development. One estimate from PayPal is that they shorten this process by 50%
   2. Containers also, as reported by MetLife, reduce infrastructure costs by 65% to 75%. Thus, containers also impact on the costs of using infrastructure, a separate area for this analysis. This impact is difficult to assess, because by shortening the software development cycle, containers let firms do more productive and profitable business, so there is very likely more overall demand for infrastructure resources, although the cost per service declines. 
   3. These impacts are reported in cases where Pods, the next evolution in the use of containers, are not considered in the measurement of benefits.

We focus on the following areas to estimate the benefits of cloud computing, AI and ML on the economy. These areas are

1. Software development including the use of Pods as well as containers:
2. Infrastructure costs and changes, focusing on changes in compute power as well as in data center architecture, management and controls.
3. Data analytics costs and innovations.

Within each of these areas, we have the following view of the main drivers of economic benefits. We identify a set of “driving factors” that generate economic changes and establish a series of “subsidiary factors” that we assume have less of an economic impact.

Software development:

a. “Driving Factors”

i. Containers –
ii. Pods, or containers bundled together locally
iii. Kubernetes – the management of hybrid clouds
iv. Open Source architectures and GitHub use
b. “Subsidiary Factors”
i. Coordination across Hybrid Clouds using Istio
ii. Microservices architecture – Machine Language code can be a collection of Microservices or “Lego-like” building blocks.
iii. Scheduling and Workflow management
iv. DevOps and Continuous Innovation/Continuous Development (CI/CD)
v. Engineer features and build models – this lets development groups to interpret model results and their accuracy.
vi. Prediction delivery

2. Infrastructure Changes

a. “Driving Factors”

i. Containers (and Pods)
b. “Subsidiary Factors”
i. Changes in compute infrastructure including the use of Graphical Processing Units (GPUs)
ii. Changes in data center infrastructure
iii. Changes in communications infrastructure through virtualization of networks.
iv. Programmable data planes.

3. Data Analytics – the software to run data analytics, algorithms, training of algorithms, and the extraction of key findings.

a. “Driving Factors”
i. Creation of machine learning (artificial intelligence) software/algorithms to analyze data. This is to measure how much the addition of “intelligence” to data analytics changes performance.
ii. Training of algorithms to improve their ability to evaluate data and extract findings.
b. “Subsidiary Factors”
i. Data collection
ii. Data verification
iii. Feature extraction
v. Configuration
v. Monitoring of the serving infrastructure
vi. Machine response management
vii. Process management for tools and oversight
viii. Monitoring the health of AI pipelines and monitoring performance scans

Organizing how we quantify impacts in this way simplifies what needs to be estimated to evaluate the main impacts of cloud, AI and ML. We define seven “driving factors” that represent the factors that we believe have the largest impact on efficiency, output and jobs.
This approach for measurement should help us develop an estimate of the economic benefits for each of the three areas we have defined above.
Initial Steps

We would evaluate the size of the benefits related to the “driving factors” identified above. We plan to do this using case studies that report the benefits of each factor and selected interviews with experienced professionals who are familiar with the factors.

One challenge is quantifying benefits that are usually described in qualitative ways. For instance, we know that containers reduce application downtime and associated costs, but we do not have a good indication of the savings for a typical firm. In addition, we know that containers have benefits beyond reduced application downtime costs, such as improved application quality and reduced defects. To add up all of the benefits, we probably need to do a series of in-depth interviews with people who use containers and might be able to estimate some of the impacts we want to measure.

Another approach would be to establish a reasonable “ballpark” estimate of the likely benefits of containers for software development as well as infrastructure. This could help us prepare two different forecasts of benefits, one high-level and another low-level that could set a reasonable set of boundaries on the possible benefits.