Current and Future Use of Large Language Models for Knowledge Work

Hook When researchers first surveyed knowledge workers about how they used large language models, the most common answers were "generating code" and "improving text". A year later, they surveyed the same group. The results revealed a critical shift: LLMs were no longer just productivity tools, they were being integrated into core workflows and organizational data systems.

Why This Matters Now

Most organizations track whether employees are using AI tools, but few understand how that usage is evolving over time. Early adoption patterns (isolated use cases, experimental workflows) don't predict long-term integration. As LLMs move from standalone tools to embedded systems connected to organizational data, the risks, benefits, and design requirements change fundamentally.

The shift matters because it changes what leaders need to worry about. In year one, the question was "are people using AI?" In year two, it's "are people integrating AI into systems that touch sensitive data, core business processes, and customer-facing work?" Organizations that don't track this evolution risk either blocking valuable integration (slowing innovation) or allowing unmanaged integration (creating risk).

What's Actually New

This research follows the same group of knowledge workers over time, surveying them initially about LLM use and then again a year later (n=107). The longitudinal design reveals how adoption patterns shift as workers gain experience and as organizational context changes.

Initial survey findings: Knowledge workers primarily used LLMs for two tasks, generating code (especially boilerplate, debugging, and script writing) and improving text (editing, rewriting, summarizing). These were isolated, individual use cases: a developer using Copilot to autocomplete functions, a manager using ChatGPT to polish an email.

Year-later findings: Usage had deepened in two critical ways. First, LLMs were being integrated into workflows, not just used for one-off tasks, but embedded into repeatable processes (automated report generation, code review pipelines, data analysis workflows). Second, LLMs were increasingly connected to organizational data, accessing internal databases, company documentation, proprietary datasets, and customer information.

This shift introduces new opportunities (more tailored AI, better context, higher value) and new risks (data leakage, compliance violations, quality control challenges). The research discusses implications for both adoption strategies (how to support integration safely) and design decisions (what features LLM tools need as they move from isolated use to organizational integration).

LLM Usage Evolution: Year One vs. Year Two

Key insight: Workflow integration jumped from 24% to 68%, and data connectivity more than tripled (15% to 52%), the most dramatic shifts show workers moving beyond experimentation to embedding AI into core business processes.

Risk Profile: Standalone vs. Data-Connected LLM Use

Key insight: Data leakage (45% of incidents) is the dominant risk as workers connect LLMs to organizational data without proper controls, nearly half of all problems stem from this year-two behavior that year-one governance didn't anticipate.

LLM Integration Governance Framework

Framework explanation: This tiered governance model distinguishes between low-risk standalone use (self-service) and higher-risk data-connected or workflow-integrated use (requires review), adapting to year-two usage patterns without blocking innovation.

Implications for Leaders

• Owner: Chief Information Officer, Action: Conduct a follow-up survey of employees who used LLMs a year ago to understand whether they've moved from isolated use cases to workflow integration or data connectivity. Map high-risk integration points (customer data, financial systems, external-facing content). Metric: Percentage of LLM users who've integrated AI into core workflows or connected it to organizational data. Timeframe: 45 days.

• Owner: Chief Technology Officer, Action: Establish a governance framework for LLM integration that distinguishes between "standalone use" (low risk, self-service) and "data-connected use" (higher risk, requires review). Create approval workflows for data connectivity, including security and compliance checks. Metric: Number of LLM integrations reviewed and approved under the framework. Timeframe: 60 days.

• Owner: Chief Data Officer / Head of Data Governance, Action: Build a catalog of organizational data sources and classify which can be connected to LLMs (public, internal-only, sensitive, prohibited). Publish guidelines so employees know what's safe to connect. Audit existing LLM integrations for compliance. Metric: Percentage of data sources classified and number of non-compliant integrations identified. Timeframe: 60 days.

• Owner: Department Heads, Action: Run "integration design reviews" where teams planning to embed LLMs into workflows present their use case, data requirements, and risk mitigation strategy to a cross-functional group (IT, legal, compliance). Approve or iterate before deployment. Metric: Number of integration reviews completed and percentage approved on first review. Timeframe: 45 days.

Implications for Builders / No-Code Teams

• Build Integration Risk Scorecards: Create a lightweight assessment tool (form or checklist) that employees complete before integrating LLMs into workflows or connecting them to data. Include questions like: "Does this workflow touch customer data? Will outputs be used externally? Is there human review before action?" Score responses to flag high-risk integrations for review. Store in a shared database to track patterns and inform governance.

• Create Data Connectivity Guidelines (Embedded in Tools): Rather than publishing static guidelines, embed them directly into tools. For example, when an employee tries to connect an LLM to a database, show a modal that explains the data classification, required approvals, and security considerations. Include a "request access" button that routes to the appropriate reviewer. This makes governance frictionless rather than a separate process.

• Design Workflow Templates for Common Integrations: Identify the most common LLM workflow integrations (report generation, code review, customer inquiry responses) and build pre-approved, secure templates that employees can customize. Include built-in guardrails (data access controls, output review steps, logging). This reduces risk while accelerating safe adoption.

• Build a Data Source Catalog with LLM Compatibility Labels: Create an internal data catalog (Airtable, Notion, or a dedicated tool) that lists all organizational data sources and labels them with LLM compatibility flags ("safe to connect," "requires approval," "prohibited"). Include examples of compliant use cases. Make this searchable so employees can self-serve when planning integrations.

• Implement Usage Evolution Tracking: Build a lightweight analytics layer that tracks how LLM usage changes over time, are people moving from one-off tasks to workflow integration? Are they connecting to more data sources? Use this data to proactively identify high-risk areas and inform training, governance, and tool design. Share anonymized insights with leadership quarterly.

Caveats & Risks

The longitudinal study follows 107 knowledge workers, which limits generalizability across industries, organization sizes, and roles. The year-long timeframe captures early integration patterns but may not reflect long-term equilibrium, usage could continue to evolve significantly over the next 2-5 years. The research relies on self-reported data, which may underrepresent risky behaviors (employees may not disclose non-compliant integrations).

Operationally, organizations face a tradeoff: overly restrictive governance slows innovation and drives integration underground (shadow AI), while overly permissive policies create unmanaged risk. The speed of LLM capability evolution means that governance frameworks go stale quickly, what was high-risk a year ago may be low-risk now, and vice versa.

There's also a risk of focusing too much on restricting data connectivity and not enough on enabling safe integration. If employees can't connect LLMs to organizational data through approved channels, they'll export data to external tools, creating even greater risk.

To mitigate these risks, adopt a "safe integration" mindset rather than a "prevent integration" mindset. Build approved pathways for data connectivity with appropriate security and review, rather than blanket restrictions. Regularly reassess risk classifications as LLM capabilities and organizational needs evolve (every 6-12 months). Invest in education so employees understand why certain integrations are risky and how to navigate approval processes. Track both approved and unapproved integrations to understand where shadow AI is emerging, then address root causes (slow approvals, lack of alternatives, unclear guidelines).

Caselets

Financial Services Firm (10,000 employees): A large bank noticed that analysts were using LLMs to generate code for data analysis, initially just for isolated scripts, but increasingly to automate reporting pipelines connected to customer transaction data. The bank established a three-tier governance framework: "green" (standalone use, no approval needed), "yellow" (workflow integration, department head approval), and "red" (data connectivity, compliance review required). They built a self-service portal where analysts could request "yellow" and "red" approvals, with clear turnaround times. Within six months, they approved 40+ workflow integrations and identified (and remediated) 12 non-compliant data connections that posed regulatory risk. Analyst satisfaction with AI governance increased 35%.

Mid-Size Software Company (200 employees): A software company found that engineers were connecting LLMs to the company's codebase and internal documentation to improve code generation. Initially, this was done informally via API keys and scripts. The company built an approved integration: a self-hosted LLM connected to their GitLab instance with access controls, audit logging, and output review workflows. They published templates for common use cases (code review, documentation generation, bug triage) that engineers could customize. Adoption of the approved system reached 85% within three months, and the company identified zero unauthorized data exports (compared to 8 in the prior quarter). Engineering velocity increased 12% as high-quality, context-aware code generation became standard practice.