This course was developed as a MEST Micro-Certificate. MEST Micro-Certificates offer short, self-paced learning opportunities through focused training modules and webinars on specialized technical topics. Participants who successfully complete the program requirements earn a certificate of completion.
These programs are designed for U.S. citizens, including Department of War and other government personnel, government-affiliated employees, industry professionals, college students, and faculty.
Large Language Models (LLMs) are a class of machine learning models that have revolutionized the field of Natural Language Processing (NLP). LLMs are capable of understanding human-like text in a wide range of applications, including text classification, translation, and even intelligent chatbots. This course provides an introduction to the Transformer architecture, which is the foundation of many state-of-the-art LLMs, originating in the models BERT and GPT. The course covers the development of language models, both in the general capabilities sense and in the specific areas of training and construction. A set of homework assignments are included to give experience with the fundamental training and use of LLMs, both in the development-side context and the foundation model view.
This course introduces the concept of the language model, a machine learning model that can understand (via classification, translation, or emergent few-shot and zero-shot in-context learning) tasks on natural language text data. The course begins with a brief summary of the Recurrent Neural Network (RNN) and Long-Short-Term-Memory (LSTM) architectures, as background for a greater discussion of the Transformer architecture. We identify the Transformer’s place as the foundation of many state-of-the-art Large Language Models (LLMs), including BERT and GPT, and discuss the development of language models in the general capabilities (use in practice) sense and in the specific areas of training and construction. The course is divided into three modules:
Unit 1: Introduction to Large Language Models (LLMs): Jumps into the Transformer architecture, via brief summary of the Recurrent Neural Network (RNN) and Long-Short-Term-Memory (LSTM) architectures. Uses this as a springboard to introduce the development of language models and their replacement of the NLP pipeline with models showcasing emergent capabilities.
Unit 2: Introduction to IEA-Plot – Building an AI Assistant in Practice: Showcases a real research tool demonstrating application of modern large language models to silicon test data analytics tasks without sacrificing trust and reliability of displayed results. Shows the internal “thought process” of the tool which maintains reliability, and how that behavior was built from less reliable and domain-specific tools like chat-capable language models.
Unit 3: Origin of Language Models – Self-Supervised Learning: Thoroughly explains the training methods used to create BERT and GPT large language models and their most effective predecessors. Shows how these models process the meaning of their inputs internally, and where their understanding of that meaning is developed.
Format: Online, asynchronous
Platform: nanoHUB Learning Management System
Estimated Time: 15-18 hours
Level: Advanced Undergraduate/Graduate Student
Prerequisites:
An understanding of the high-level programming language Python and basic mathematics (algebra, linear algebra, trigonometry, derivatives) is assumed.
Knowledge of the fundamental units of microelectronics fabrication (wafers, die, lithography) is helpful but not required.
MEST Micro-Certificate: Introduction to Machine Learning is highly recommended.
Instructor: Professor Li-C Wang, University of California, Santa Barbara
Course Objectives:
Explain the evolution of language model architectures from RNNs to Transformers
Compare the capabilities of traditional NLP pipelines with LLM-based approaches
Analyze the implementation of IEA-Plot for silicon test data analytics
Describe the self-supervised learning process underlying BERT and GPT
Evaluate the trust and reliability aspects of AI assistants using LLMs
Request Course Enrollment
To enroll in this course, you must have a nanoHUB account. Create one here: https://nanohub.org/register/
Request membership in the MEST group in nanoHUB at https://nanohub.org/groups/mest