AI (Large Language Model) Technology Principles and Applications: Empowering ITBP Team Capability Enhancement

Jeffrey Jiao, AI and Digital Transformation Practitioner, Former McKinsey Consultant

2025/10/09 10:00 Meeting update on requirements:

Add retail FMCG (Fast-Moving Consumer Goods) cases, expected to provide insights for drug-to-consumer (to C) sales;

Basic concept learning materials are provided one week before class so that students can enter the classroom quickly.

2025/09/18 10:30 Meeting update on requirements:

AI (Large Language Model) Technology Principles and Applications: Empowering ITBP Team Capability Enhancement

Jeffrey Jiao, AI and Digital Transformation Practitioner, Former McKinsey Consultant

Update memo of requirements:

2025/10/09 10:00 Meeting update on requirements:

Add retail FMCG (Fast-Moving Consumer Goods) cases, expected to provide insights for drug-to-consumer (to C) sales;

Basic concept learning materials are provided one week before class so that students can enter the classroom quickly.

2025/09/18 10:30 Meeting update on requirements:

Training objectives:

Align the ITBP team (9 people) on AI cognition and conceptual language system;

To become Merck's pioneer in China in the field of AI;

Demand understanding:

Establish a global perspective: study the differences and comparisons between Chinese and American representative companies in AI understanding, development direction, product development route and methodology;

Industry Practical Experience: Analyze lessons learned from AI applications across different industries (domestic, international, and foreign-funded); examine the AI strategies of major Chinese tech companies (Baidu, Alibaba, Tencent, ByteDance, etc.) and central state-owned enterprises; and provide reference frameworks for large model selection in B2B enterprises (particularly healthcare).

Other requirements:

Hope to be knowledge intensive, theoretical and interesting;

In the morning, everyone is energetic and has a higher knowledge density, and in the afternoon, there is more interaction;

From 15:00 to 15:30, invite German colleagues to share for half an hour on planning, strategy, etc.;

Materials should be in English, with bilingual (Chinese-English) format recommended.

program objective

Help the team to deepen the understanding of AI technology, realize the transformation from theory to practical application, and be able to use AI to solve work pain points; cultivate technical capabilities beyond theory, and non-technical colleagues can also master basic practical coding skills.

Build a systematic AI technology capability framework for the ITBP team to facilitate the implementation of enterprise digital transformation.

Course Objectives

The entire ITBP team (both technical and non-technical backgrounds, friendly to beginners)

Duration of instruction

A total of 6 hours (theoretical explanation 35%+ practical operation 45%+ interactive discussion 20%)

course content

Part 1: Global Vision and Technology Frontiers

Global vision for AI development

Global development of AI and China-US comparison

Global AI development status and trends

Technological development speed: The evolution speed of artificial intelligence is far faster than expected. Experts predict that general artificial intelligence (AGI) may be realized in the next 2-6 years, and the technological progress cycle will be shortened from the century of the industrial age to the month and week level of the AI era.

Performance breakthrough: The performance of the AI system continues to break through in the complex benchmarking, and the scores of MMMU, GPQA and SWE-bench increased by 18.8, 48.9 and 67.3 percentage points respectively within one year.

Market size: The total size of China's AI market is expected to reach 51.13 billion US dollars in 2025, and the market size of AI-related markets will exceed 3.8 trillion yuan.

Comparison of AI development paths between China and the US

The "elite" route vs the "industrialist" route

dimension

America

China

idea of development

Lab elitism, technology-driven

Industrial pragmatism, scenario driven

technology roadmap

Closed source monopolies, dominated by private companies

Multiple open source, government guidance and industrial integration

Areas of strength

Base model, underlying architecture, open source ecology

Application scenarios, data resources, industrialization speed

Representative enterprises

OpenAI、Google、Anthropic

Alibaba, Huawei, Baidu, Tencent

International strategy

Technical protection, export restrictions, building alliances

Open source sharing, promoting products and open source models

The changing technology gap and the rise of open source power

The performance gap between China and the US has narrowed significantly: in benchmarks such as MMLU, MMMU, MATH and HumanEval, the performance gap between China and the US has narrowed significantly from 17.5, 13.5, 24.3 and 31.6 percentage points at the end of 2023 to only 0.3, 8.1, 1.6 and 3.7 percentage points at the end of 2024.

The rise of open-source models: China is accelerating the construction of an "independently controllable and open collaborative" industrial system, pooling resources through a new national system for technological breakthroughs. AliTongyi Qianwen has become the world's largest open-source large model, with its derivative models surpassing 100,000 in number, outperforming the American llama.

Breakthrough of DeepSeek: On January 20,2025, DeepSeek released R1 reasoning model, which achieved performance equivalent to OpenAI o1 at a very low cost (only $0.55 per million input tags, while OpenAI o1 is up to $15), which was called "the Android moment in AI".

Advances and breakthroughs in large model technology

Three major changes in technological development

From "aesthetics of violence" to "meticulous cultivation": the competition of large models shifts from the comparison of parameter scale to the improvement of efficiency and application;

From "heavy training" to "heavy reasoning": The reasoning efficiency is significantly improved through reinforcement learning and knowledge distillation technology;

From "single mode" to "multi-mode fusion": seamless fusion of text, image, audio and video.

Major technological breakthroughs in 2024-2025

Mixed expert architecture (MoE): GPT-5 has 512 experts, and the activated parameters account for only 7%, greatly improving efficiency;

Breakthrough in reasoning ability: OpenAI's o3 outperforms some human experts in fields such as mathematics and programming; Google's Gemini 2.0 Lightning Thinking demonstrates amazing problem-solving ability in physics and engineering problems;

Multimodal fusion: CLIP++ model achieves four-mode joint representation of text, image, audio and video, and the cross-mode retrieval accuracy is 91.3%;

Edge AI development: The global edge AI market size will reach 121.204 billion yuan in 2024 and is expected to grow by 29.49% CAGR to 571.486 billion yuan in 2030.

The rise of AI agents (Agents)

Autonomous improvement: AI agents can handle complex multi-step tasks without human intervention, from "passive response" to "active service";

Commercial Year: 2025 will be the "commercial year" of AI Agent. By 2028,15% of daily work decisions are expected to be completed by Agentic AI;

Enterprise applications: Deloitte predicts that agents will be able to support the work of supply chain managers, software developers, financial analysts and others;

Computing power infrastructure and hardware ecology

China's computing power scale: China's intelligent computing power scale will reach 725.3 EFLOPS in 2024 and is expected to reach 1037.3 EFLOPS in 2025, an increase of 43% compared with 2024.

Cost of computing power: Training costs for a single model plunged from $12 million in 2022 to $850,000.

Green computing power development: With the popularization of large models, green solutions such as liquid cooling technology and edge computing are developing rapidly.

Comparison of hardware ecology between China and the US:

The United States has a clear advantage: The NVIDIA H100 achieves 1,979 TFLOPS in half-precision (FP16) computing power, which is 7.7 times that of Huawei's Ascend 910B, and its video memory bandwidth is 336% higher.

CUDA ecological barrier: Nvidia A100/H100 series occupies 90% of the global AI training chip market, and the CUDA ecosystem forms an insurmountable technical stickiness.

China's Breakthrough Strategy: Focusing on "Scenario Customization + Process Innovation", Huawei Ascend 910B achieves 14nm process mass production through Chiplet technology, with an energy efficiency ratio exceeding 30% in smart city video analysis.

Core knowledge of large models

Core concepts (the concepts and methods involved in the data set to the pre-training phase)

It covers all the conceptual principles and practical processes involved from data set construction to base model formation, and is expected to achieve the level of "mastering first principles" for large models.

Concepts: What is a data set? How does it differ from algorithms or computing power? Where are the data set scarcity issues? Why did Transformers emerge? What is the self-attention mechanism? What exactly does pre-training aim to achieve?...

Part 2: Industry practice and interactive discussion

Core knowledge points of large models

Core concepts (emphasis on the concepts and methods involved in "vertical model" training)

Note: The afternoon session will feature more interactive activities, where participants can apply concepts, principles, and methodologies from large models to engage in open discussions. Topics will include insights from industry leaders, future business trends, healthcare industry strategies, and the positioning and value delivery of IT-BP solutions...

The time was set around 3:00PM and a German colleague was introduced to share.

Introduction to large model integration and Agent Design

What is an AI Agent

Popular definition, attributes and classification of intelligent body

Perceive the environment and act to achieve your goals

Attribute: autonomy, goal orientation, perception, initiative

Classification: simple reflex, model reflex, based on target, based on utility, learning type

General architecture of intelligent agents

AI Agent= large model + memory + planning + tools

Intelligent agent development framework

Select development platform: Internet platform, local deployment

Model selection: based on professional attributes, data attributes, multimodal attributes, etc

Service orchestration: prompt engineering, single Agent/multi Agent, component invocation, etc

Core competency development: workflow, memory, knowledge, conversational experience, etc

Release and maintenance

Application of prompts in the process of building agents

Understand the task prompts

Task decomposition, task definition, introduction of examples

Task planning and execution

Task chain design, multiple rounds of task execution, execution feedback adjustment

Match information retrieval

Limited information sources, multi-level retrieval and dynamic retrieval

User interaction design

Multimodality, context

NL2SQL prompt application

definition

NL2SQL, also known as Text2SQL, is the conversion of a user's natural language into SQL statements so that the user can retrieve the data from the database.

Challenges in practical use of NL2SQL

The ambiguity and uncertainty of natural semantics, the complexity of relationships between data tables, the integrity of data, and similarity.

Zero-shot and Few-Shot

NL2SQL prompt element

Tasks, reference information, output requirements, examples, input and output items for this time

Common prompt cases

Text prompt method, role play method, code prompt method, instruction fine-tuning prompt method, context learning

Cutting-edge exploration

C3、CoT、DIN-SQL

RAG prompt application (vs context project)

RAG architecture

Question analysis and continuation

Information retrieval and integration

Optimization strategies for generating answers

Limit and guide the generation model

Industry pioneers

AI's iPhone Moment: From Cloud Native to AI Primitive

Stanford 2025 Artificial Intelligence Index Report

Cloud native vs. AI native

The connotation of AI native: AI reshapes software and enables intelligent industry

Huawei, Tencent and other enterprises' AI architecture evolution ideas

Sierra's innovation (founded by OpenAI's board chairman): A new way to 'connect' with customers

Zhong Mobile AI Lingxi: 1 billion users have an AI assistant

Industry application practice and lessons learned

Cross-industry application case sharing

manufacturing industry ：

A steel plant used a large model to analyze equipment data, predicting machine failures three days in advance and reducing maintenance costs by 40%.

China has built 1,200 smart factories, with 72 "lighthouse factories" accounting for 42% of the global total. AI has optimized production processes, reducing energy consumption by 15%.

Tesla Optimus Gen2 robot has been applied in factory production, and man-machine collaboration will become an important mode of intelligent manufacturing in the future

banking business ：

Ant Group's risk control brain 3.0 achieved a credit fraud identification accuracy of 99.993% (an increase of 3 orders of magnitude compared with 2022)

An insurance company used a large model to automatically identify injury photos, reducing the time to settle claims from three days to 10 minutes, and customer satisfaction doubled

Cross-border payment and clearing efficiency has been improved by 22 times, supporting RCEP regional second-level arrival

Medical care:

The diagnostic accuracy of IBM Watson has reached 98.7%, surpassing the average for senior physicians

The AI-assisted cervical cytology diagnosis system developed by Tencent and Ambipin has an accuracy rate of more than 95 percent and has been deployed in more than 7,000 hospitals

Hengrui Pharmaceutical applied AI to new drug research and development through the "Bai Cheng Intelligent Drug Platform", shortening the research and development cycle by 30%

Educational field ：

The introduction of AI tools in primary and secondary schools in Beijing has increased the efficiency of English classes by 30%

A middle school used a large model to analyze students' mistakes and automatically generate targeted exercises, raising the class average by 15 points

Duolingo, the gamified language learning app in the US, has surpassed 100 million monthly active users, while Khan Academy's AI tutors guide inquiry-based learning

Retail field ：

Meituan, China Mobile's掌厅to C, Sierra (OpenAI board of directors) and other retail innovation achievements

Summary of application experience and lessons learned

Avoid blindly pursuing "big and complete": A food factory spent millions of yuan to train a general model, only to find that the industry terms were not recognized accurately. We should first build small models in vertical fields and then expand gradually.

Data is more important than algorithms: A logistics company trained its path planning model with publicly available online data, but the actual results were not satisfactory. Real business data should be accumulated, even if it is only Excel tables.

Employees are not rivals, but helpers: a bank pushed smart customer service, which was resisted, and then trained tellers to use AI to answer professional questions, improving service efficiency by 50%.

The red line of security should not be touched: A hospital was punished by the regulatory authorities for directly uploading patients' CT images to the public cloud. Sensitive data must be deployed on-premises.

Analysis of AI planning by China's major companies and state-owned enterprises

AI strategy of Internet giants

Ali: Tongyi Qianwen has become the world's first open source large model, and the number of derivative models has exceeded 100,000. It is expected to invest 380 billion yuan in AI and cloud computing in the next three years.

Baidu: Wenxin Yiyang is commercialized through search, e-commerce recommendation and other scenarios.

Tencent: The average reasoning delay of the hybrid model in the field of game NPC interaction is reduced by 35%.

ByteDance: Ocean Engine enables automatic generation of advertising scripts, increasing e-commerce conversion rates by 35%.

Characteristics of AI applications in central state-owned enterprises

Focus on security and control: Prioritize local deployment to ensure data security

Scenario-driven: Focus on ROI validation starting from specific business pain points

Integration of industry, academia, and research: Collaborating with research institutions to develop industry-specific large models, such as the 'Xingyu 3.0' astronomical model developed by the National Astronomical Observatories of the Chinese Academy of Sciences based on Qwen.

Reference for Model Selection of Enterprise B (Medical)

Points to consider:

In-house vs. Outsourcing: Internal and external collaboration models

Base model + vertical category: How high is the barrier to medical professional training models?

Ownership interests, security and compliance of data exchange: data privacy computing, federated learning, GDPR and other regulations

Ensuring interpretability, localization (localization), AI for Science...