Started Aug 17, 2020 02:00PM UTC   •   Closed Dec 07, 2020 09:59PM UTC

What will the ratio of AI publications to machine learning research job postings be for Apple, Amazon, Facebook, Google, and Microsoft, combined, in 2020?

Topics

Context. Changes in the ratio between publications and research activity indicates either a change in the type of research occurring or the percentage of research that's published. If a research community is becoming less open, for example, quantity of publications might not fully reflect its level of research activity. Measuring research activity is less straightforward than measuring publications, however. For this question, we use job postings for research roles as a proxy measure of research activity. 

For more on this metric, see the accompanying Foretell metric analysis: "The U.S. AI Industry’s Demand for AI-Skilled Researchers Is Increasing, but Not Its AI Publications."

Data and resolution details. This question resolves based on Dimensions (publications) and Burning Glass (job posting) data. We classified publications as AI/ML-relevant or not using a predictive model trained on arXiv publication data, where a publication is relevant if it’s categorized on arXiv under any of artificial intelligence, machine learning, computer vision, computation and language, multiagent systems, or robotics. To read more about this method, see “Identifying the Development and Application of Artificial Intelligence in Scientific Text.” For Burning Glass job postings, we included a job posting if Burning Glass included machine learning among the required skills and categorized the occupation as “Researcher / Research Associate.”

This question resolves based on data available as of January 1, 2021. Restricting data to what's available during the year in question has a mild affect on the historic data. For 2019, it changes the value from 0.73. to 0.57, for 2018 from 0.52. to 0.44, and for 2017 from 0.78 to 0.66.

As of October 15, 2020, the value for 2020 is 0.85.

 

The data underlying the graph is here.

This question is a metric for the following scenarios:

To suggest a change or clarification to this question, please select Request Clarification from the green gear-shaped dropdown button to the right of the question.

Resolution Notes

The answer is 0.94: 724 publications to 767 job postings.

Possible Answer Correct? Final Crowd Forecast
Less than 0.3 4%
Between 0.3 and 0.6, inclusive 11%
More than 0.6 but less than or equal to 0.9 38%
More than 0.9 but less than or equal to 1.2 40%
More than 1.2 7%

No. of
Forecasters

50

Forecast
Count

66

Crowd Brier Score

0.2132


Most Accurate

Relative Brier Score

1.
-0.231
2.
-0.206
3.
-0.198
4.
-0.17
5.
-0.162

Tip: Mention someone by typing @username