Creating value from big data in the investment management process: A workflow analysis
The report analyses how investment professionals use AI and big data, noting moderate but rising adoption, multihoming across tools, and key challenges including skills gaps, data quality, and model opacity. It highlights organisational priorities such as upskilling and workflow automation to enhance efficiency and decision making.
Please login or join for free to read more.
OVERVIEW
Introduction
The report examines how investment professionals use AI and big data in workflow activities, assessing risks, challenges, and opportunities. It draws on a global survey of CFA Institute members (Feb–Mar 2024; 104–569 responses; margin of error ±4%) and roundtables with executives, practitioners, learning specialists, and regulators across 10 sessions. Respondents were predominantly experienced professionals (average age 44; 17 years’ industry experience), mainly from the Americas (48%), EMEA (32%), and Asia-Pacific (20%). Analytical and investment decision-making roles formed 59% of respondents.
AI and big data applications in investment management
AI is defined as technology that enables machines to mimic intelligent human behaviour, supported by machine learning, deep learning, and neural networks. The report outlines previous CFA Institute work on AI adoption and the need for updated analysis given rapid technological development, including Python’s expanding capabilities and the integration of generative AI. Key research questions relate to how AI is used, drivers of adoption, obstacles, regulatory concerns, and opportunities.
How do investment professionals use AI technologies in the workplace?
Technological intensity was moderate: 20–39% of professionals used GenAI, Python, SQL, other programming languages, or other visualisation tools, yet frequency of use was high, with most technologies used daily. Other visualisation tools such as Tableau and Power BI showed moderately high intensity, with 47% using them daily.
Multihoming was common. Professionals frequently combined Python, SQL, GenAI, and visualisation tools in workflows.
Python was used largely for data analysis (34%) and data visualisation (39%).
SQL and other languages were most used in risk management (24% and 15%).
GenAI use was concentrated in business development (22%) and industry/company analysis (20%).
ChatGPT was the dominant GenAI tool (used by 86% of GenAI users), with Microsoft Copilot the next most common.
Regional usage patterns were similar: AI tools were most used for data analysis (Americas 25%, APAC 23%, EMEA 24%) and data visualisation (Americas 25%, APAC 32%, EMEA 23%). Use was highest among Analytical and Investment & Decision-Making roles, with daily or weekly usage rates exceeding 74% and 96% respectively.
AI technologies in investment management: Organisational issues
Implementation maturity varied: 23% of organisations were developing or procuring AI tools, 21% reported decentralised efforts, and 15% had coordinated firm-wide applications. Only 9% reported extensive adoption. Key risks were data privacy (50%), model non-explainability (46%), and compliance risks (35%), with EMEA respondents additionally highlighting vendor reliance and operational risks.
Roundtable participants emphasised data privacy risks arising from LLM training, the difficulty of validating opaque models, and concerns regarding algorithmic appreciation, which may increase user overconfidence. Regulatory challenges included fragmented global policy approaches and insufficient clarity in AI frameworks.
Organisational challenges included a shortage of AI/big data talent (41%), data quality issues (41%), and regulatory constraints (32%). Skills gaps stemmed from limited hybrid finance-technology expertise. Executives noted the complexity of ML model governance compared with traditional quantitative models. Smaller regulators faced budgetary constraints in acquiring SupTech capabilities.
Organisations were responding through workflow automation (40%), upskilling staff (34%), increased platform integration (32%), and strategic focus on AI and analytics (34%). Only 6–11% expected headcount reductions.
AI technologies in investment management: Individual perspectives
Professionals expected AI to enhance 51% of workflows and replace 31% within five years, with APAC showing the narrowest enhancement–replacement gap. Over 75% planned to develop technical skills to remain relevant; 53% planned to strengthen soft skills. Skill-building interest was highest in GenAI (87%), data science (63%), and data visualisation (61%), while proficiency levels were low (≤10%).
Harnessing the power of artificial intelligence for enhanced decision making and efficiency
AI adoption offers opportunities in democratising financial advice, improving collaboration, optimising investment strategies, detecting fraud, and enhancing regulatory oversight. Examples include robo-advice growth, Python-enabled collaborative forecasting, AI-driven trading models (e.g., Bridgewater, BlackRock), blockchain-based fraud detection, and AI-enabled legal document analysis (e.g., JPMorgan’s COiN). Regulators can use AI for predictive risk monitoring, real-time surveillance, and privacy-preserving data analysis (e.g., federated learning, SMPC).
Conclusion
AI and big data use in investment management is rising, though challenges persist relating to skills, data governance, model explainability, and regulatory coherence. Firms are prioritising automation, integration, and upskilling to capture efficiency and decision-making benefits, while professionals recognise the need to improve technical capability to remain competitive.