Course · 12 modules · 95 lessons · 672 min

Natural Language Processing

Text preprocessing, representation, sequence models, NLP tasks, information extraction, and multilingual NLP.

Foundations Of Language

·Ambiguity in LanguageLexical, syntactic, semantic, and referential ambiguity -- the core challenge that makes NLP hard, illustrated by why "I saw her duck" has at least five interpretations and how systems learn to resolve them.10 min→·Levels of Linguistic AnalysisThe hierarchy from phonology to pragmatics -- each level adding structure that NLP systems must capture to move from raw sound or text to genuine understanding.7 min→·MorphologyHow words are built from morphemes -- inflection, derivation, and compounding that affect meaning, and whose cross-linguistic variation profoundly shapes tokenization and NLP system design.7 min→·Pragmatics and DiscourseMeaning beyond the literal -- context, implicature, speech acts, Gricean maxims, and discourse coherence that determine what speakers actually communicate versus what their words technically say.8 min→·SemanticsThe study of meaning -- from word senses to compositional sentence meaning to logical representations, and the distributional hypothesis that reshaped how NLP systems learn what words mean.8 min→·Syntax and GrammarRules governing sentence structure -- phrase structure grammars, dependency relations, constituency trees, and the Chomsky hierarchy that defines the computational complexity of parsing.8 min→·Text as DataTreating language as structured data -- corpora, annotation schemes, inter-annotator agreement, and the paradigm shift from hand-crafted features to learned representations that transformed how NLP systems consume text.10 min→·What Is NLPNatural Language Processing is the interdisciplinary field at the intersection of linguistics, computer science, and artificial intelligence that enables machines to understand, generate, and reason about human language.7 min→

Text Preprocessing

·Data Annotation and LabelingCreating labeled NLP datasets through systematic annotation schemes, measuring inter-annotator agreement, managing crowdsourced labor, and applying active learning to minimize the high cost of human labeling.9 min→·Regular Expressions for NLPPattern matching as the workhorse of text preprocessing -- defining formal string patterns with a concise syntax to search, extract, validate, and transform text in NLP pipelines.6 min→·Sentence SegmentationDetecting sentence boundaries in running text despite the ambiguity of periods, which serve triple duty as sentence terminators, abbreviation markers, and decimal points.6 min→·Stemming and LemmatizationReducing words to base forms -- stemming by crude affix removal and lemmatization by linguistically-informed morphological analysis -- to collapse inflectional variants into shared representations.6 min→·Stopword RemovalFiltering high-frequency function words (the, is, at, which) that carry little semantic content to reduce noise and dimensionality in frequency-based text representations, though modern neural models often benefit from retaining them.6 min→·Text Cleaning and Noise RemovalHandling the messy reality of real-world text -- stripping HTML, fixing encoding errors, correcting OCR artifacts, normalizing social media conventions, deduplicating, and detecting language -- before any NLP model can be reliably applied.8 min→·Text NormalizationStandardizing text through case folding, unicode normalization, accent removal, and format unification so that superficially different strings map to a single canonical form before downstream processing.5 min→·Tokenization in NLPSplitting raw text into discrete units -- words and sentences -- using rule-based, statistical, or hybrid methods, with strategies that vary dramatically across languages and domains.6 min→

Text Representation

·Bag of WordsRepresenting text as unordered word frequency vectors -- simple, interpretable, and surprisingly effective for many classification and retrieval tasks.5 min→·Contextual EmbeddingsWord representations that change based on surrounding context -- the same word gets different vectors in different sentences, resolving polysemy and capturing nuance.7 min→·Document EmbeddingsRepresenting documents as dense vectors for retrieval, clustering, and classification at scale -- from TF-IDF with dimensionality reduction to neural encoders for long text.6 min→·FastTextSubword-aware embeddings that represent each word as the sum of its character n-gram vectors, gracefully handling morphology and out-of-vocabulary words.6 min→·GloVeGlobal matrix factorization of word co-occurrence statistics producing word vectors with linear substructures -- bridging count-based and prediction-based embedding methods.6 min→·N-Gram Language ModelsPredicting the next word from the previous N-1 words using maximum likelihood estimation -- the statistical foundation of language modeling.5 min→·Sentence EmbeddingsFixed-length vector representations of entire sentences -- from simple word vector averaging to dedicated neural encoders trained for semantic similarity.6 min→·TF-IDFWeighting words by term frequency times inverse document frequency to surface discriminative terms and suppress ubiquitous ones.5 min→·Word2VecLearning dense word vectors from co-occurrence via Skip-gram and CBOW -- the embedding revolution that showed words with similar meanings occupy nearby points in vector space.6 min→

Sequence Models

·Attention MechanismAttention allows a decoder to dynamically focus on different parts of the encoder's output at each generation step, replacing the fixed-size bottleneck vector with a weighted combination of all source representations.8 min→·Bidirectional RNNsBidirectional RNNs process a sequence in both forward and backward directions, producing representations that capture both past and future context at every time step.7 min→·Convolutional Models for TextCNNs applied to NLP use 1D convolutions over word embeddings to detect local n-gram patterns, offering parallelizable computation and strong performance for text classification, though with a limited receptive field compared to recurrent models.8 min→·Gated Recurrent UnitsGRUs simplify the LSTM gating mechanism by merging the cell state and hidden state into a single vector controlled by two gates, achieving comparable performance with fewer parameters.7 min→·Long Short-Term MemoryLSTMs introduce a gated cell state that acts as a controlled information highway, solving the vanishing gradient problem that cripples vanilla RNNs on long sequences.7 min→·Recurrent Neural NetworksRNNs process sequences one element at a time, maintaining a hidden state that accumulates information from previous time steps -- the first neural architecture designed for sequential data like language.6 min→·Sequence-to-Sequence ModelsThe encoder-decoder architecture maps variable-length input sequences to variable-length output sequences by compressing the input into a fixed-size context vector, then generating the output one token at a time.8 min→

Core Nlp Tasks Analysis

·Constituency ParsingConstituency parsing decomposes sentences into hierarchical phrase-structure trees, revealing how words group into nested constituents (noun phrases, verb phrases, etc.) according to a grammar.6 min→·Coreference ResolutionCoreference resolution determines which expressions (mentions) in a text refer to the same real-world entity, linking pronouns, names, and descriptions into coherent entity chains.5 min→·Dependency ParsingDependency parsing recovers the head-modifier relationships between words in a sentence, producing a tree structure that reveals grammatical dependencies without intermediate phrasal nodes.5 min→·Named Entity RecognitionNamed entity recognition (NER) identifies and classifies spans of text that refer to real-world entities such as persons, organizations, locations, dates, and other domain-specific categories.5 min→·Part-of-Speech TaggingPart-of-speech (POS) tagging assigns a grammatical category (noun, verb, adjective, etc.) to each word in a sentence, serving as a foundational sequence labeling task for downstream NLP.5 min→·Relation ExtractionRelation extraction identifies and classifies semantic relationships between entities mentioned in text, converting unstructured prose into structured knowledge triples (subject, relation, object).6 min→·Semantic Role LabelingSemantic role labeling (SRL) identifies the predicate-argument structure of a sentence -- determining who did what to whom, where, when, and how -- bridging syntactic parsing and meaning representation.5 min→·Sentiment AnalysisSentiment analysis detects subjective opinion, polarity (positive/negative/neutral), and emotion in text, operating at document, sentence, and aspect levels of granularity.5 min→·Text ClassificationText classification assigns one or more predefined category labels to a document, sentence, or passage, serving as the most widely deployed NLP capability in production systems.5 min→

Core Nlp Tasks Generation

·Data-to-Text GenerationConverting structured data (tables, knowledge graphs, database records) into fluent natural language descriptions, bridging the gap between databases and human-readable reports.6 min→·Dialogue SystemsConversational AI systems that interact with users through natural language, ranging from task-oriented slot-filling assistants to open-domain chatbots and modern LLM-based dialogue agents.7 min→·Grammatical Error CorrectionDetecting and correcting grammatical, spelling, and usage errors in written text, progressing from rule-based checkers through classifier ensembles to neural sequence-to-sequence and LLM-based approaches.7 min→·Machine TranslationAutomatically converting text from one human language to another, progressing from hand-crafted rules through statistical phrase tables to end-to-end neural models.6 min→·Paraphrase GenerationProducing semantically equivalent but syntactically different text, enabling data augmentation, style transfer, and deeper understanding of meaning.6 min→·Question AnsweringSystems that find or generate answers to natural language questions from given context, retrieved documents, or parametric knowledge.6 min→·Text GenerationProducing fluent, coherent text from a language model using decoding strategies that balance quality, diversity, and controllability.6 min→·Text SummarizationCondensing documents while preserving key information, using either extractive methods that select important sentences or abstractive methods that generate new condensed text.5 min→

Information Extraction And Retrieval

Semantic Understanding

·Aspect-Based Sentiment AnalysisAspect-based sentiment analysis (ABSA) goes beyond document-level opinion mining to identify specific aspects of entities and the sentiment expressed toward each, enabling fine-grained understanding of opinions like "The food was great but the service was terrible."10 min→·Commonsense ReasoningCommonsense reasoning is the ability to draw on implicit world knowledge that humans take for granted -- physical intuitions, social conventions, and causal expectations -- to understand and reason about language.9 min→·Natural Language InferenceNatural language inference (NLI) classifies the relationship between a premise and hypothesis as entailment, contradiction, or neutral, serving as both a core semantic reasoning benchmark and a versatile tool for zero-shot NLP.9 min→·Negation and Speculation DetectionNegation and speculation detection identifies negated and uncertain statements in text -- determining not just what is said, but what is denied or merely hypothesized -- a capability critical for biomedical NLP, information extraction, and sentiment analysis.10 min→·Semantic SimilaritySemantic similarity measures the degree of meaning overlap between two linguistic units -- words, sentences, or documents -- providing a graded, continuous score rather than a categorical judgment.8 min→·Temporal ReasoningTemporal reasoning is the ability to identify, interpret, and reason about time expressions, event ordering, and temporal relationships in text, enabling systems to construct timelines and answer when-questions.10 min→·Textual EntailmentTextual entailment is the task of determining whether the meaning of one text (the hypothesis) can be logically inferred from another text (the premise), forming the foundation of computational semantic inference.8 min→·Word Sense DisambiguationWord sense disambiguation (WSD) is the task of determining which meaning of a polysemous word is intended in a given context, resolving one of the oldest and most fundamental ambiguities in natural language processing.8 min→

Pre Trained Models For Nlp

·BERTBERT (Bidirectional Encoder Representations from Transformers) pre-trains a deep transformer encoder using masked language modeling and next sentence prediction, producing bidirectional contextualized representations that shattered records across 11 NLP benchmarks and spawned an entire family of variants that continue to dominate NLP.7 min→·Cross-Lingual TransferCross-lingual transfer leverages multilingual pre-trained models to transfer NLP capabilities from high-resource languages (primarily English) to low-resource languages without target-language labeled data -- enabling zero-shot task performance across 100+ languages through shared representations.7 min→·Domain AdaptationDomain adaptation extends general-purpose pre-trained models to specialized domains -- biomedical, scientific, financial, legal, and clinical text -- through continued pre-training on domain corpora, producing models like BioBERT, SciBERT, and FinBERT that outperform their general counterparts by 2-10% on in-domain tasks.8 min→·ELMoELMo (Embeddings from Language Models) produces deep contextualized word representations by running a two-layer bidirectional LSTM language model, generating different vectors for the same word depending on its surrounding context -- the first major pre-trained model that bridged static word embeddings and modern transformers.7 min→·GPT for NLP TasksThe GPT series -- from GPT-1's generative pre-training with discriminative fine-tuning, through GPT-2's surprising zero-shot abilities, to GPT-3's in-context learning revolution -- demonstrated that autoregressive decoder-only transformers can perform virtually any NLP task through prompting alone, without task-specific fine-tuning.7 min→·Prompt-Based NLPPrompt-based NLP reformulates traditional NLP tasks as cloze-style fill-in-the-blank or text generation problems, leveraging pre-trained language models' existing knowledge to perform tasks with minimal or zero labeled data by converting classification into "predict the next/masked word" problems.8 min→·T5 and Text-to-TextT5 (Text-to-Text Transfer Transformer) unifies every NLP task -- classification, translation, summarization, question answering, and more -- into a single text-to-text framework where both inputs and outputs are text strings, enabling a systematic comparison of pre-training objectives, architectures, and datasets at scales from 60M to 11B parameters.8 min→·Transfer Learning in NLPTransfer learning revolutionized NLP by replacing task-specific training from scratch with a two-stage paradigm -- pre-train on massive unlabeled corpora, then fine-tune on small task-specific datasets -- reducing data requirements by 10-100x and establishing new state-of-the-art results across virtually every benchmark.7 min→

Multilingual And Low Resource Nlp

·Cross-Lingual Word EmbeddingsAligning word vector spaces from different languages into a shared space so that "cat" in English and "gato" in Spanish occupy nearby points -- enabling cross-lingual transfer without parallel corpora.6 min→·Data Augmentation for NLPGenerating synthetic training examples through techniques like back-translation, synonym replacement, and contextual generation to improve model performance when labeled data is scarce -- typically yielding 5--30% improvements depending on baseline data size.8 min→·Language Diversity and TypologyHow the structural properties of the world's languages -- word order, morphological complexity, and writing systems -- create distinct challenges for NLP systems that are overwhelmingly designed for English.10 min→·Low-Resource NLPTechniques for building effective NLP systems when labeled data is scarce -- from few-shot and zero-shot learning to active learning and cross-lingual transfer -- addressing the reality that most languages and domains lack sufficient annotated data.8 min→·Machine Translation ApproachesThe evolution of machine translation from hand-coded linguistic rules through statistical phrase tables to end-to-end neural models -- each paradigm shift dramatically improving quality and reducing engineering effort.8 min→·Multilingual NLPBuilding NLP systems that work across multiple languages -- navigating the tension between universal representations and the enormous diversity of the world's 7,000+ languages.7 min→·Multilingual TransformersPre-trained transformer models like mBERT and XLM-R that learn shared representations across 100+ languages from massive multilingual corpora, enabling zero-shot cross-lingual transfer.7 min→

Speech And Multimodal Nlp

·Automatic Speech RecognitionConverting spoken language into written text by mapping acoustic signals through feature extraction, acoustic modeling, and language decoding -- progressing from HMM-GMM pipelines to end-to-end neural systems like Whisper.7 min→·Document UnderstandingExtracting and understanding information from visually rich documents (forms, invoices, reports, tables) by jointly modeling text content, visual appearance, and spatial layout -- powered by the LayoutLM family and multimodal document representations.8 min→·Image CaptioningGenerating natural language descriptions of images by bridging visual perception and language generation -- from CNN-LSTM pipelines to attention-based and transformer models, now increasingly subsumed by vision-language foundation models.8 min→·Multimodal NLPCombining language with vision, audio, and other modalities to build systems that perceive and reason across multiple information channels -- from contrastive pre-training (CLIP) to multimodal large language models (GPT-4V, Gemini).7 min→·Speech Language ModelsUnified models that process both text and speech as token sequences, enabling zero-shot voice cloning, speech generation, and the convergence toward universal language models that handle any modality.8 min→·Text-to-SpeechGenerating natural-sounding human speech from written text, progressing from concatenative and parametric methods to neural systems (Tacotron, WaveNet, FastSpeech) that approach human-level naturalness.7 min→·Visual Question AnsweringAnswering natural language questions about images by jointly reasoning over visual and textual information -- a fundamental test of multimodal understanding that exposes the tension between genuine reasoning and superficial language bias.8 min→

Evaluation And Ethics

·Bias in NLPNLP systems absorb, reproduce, and often amplify societal biases present in training data, annotation practices, and modeling decisions, leading to systematic disadvantages for underrepresented groups.8 min→·Evaluation Metrics for NLPAutomated evaluation metrics quantify NLP system performance using formulas that approximate human judgment, each capturing a different facet of quality -- from exact-match precision to semantic embedding similarity.6 min→·Fairness in NLPFairness in NLP formalizes the requirement that language technologies perform equitably across demographic groups, using mathematical definitions that reveal fundamental trade-offs between competing notions of what "fair" means.8 min→·Human Evaluation for NLPHuman evaluation remains the gold standard for assessing NLP system quality, using structured protocols with trained annotators to judge dimensions -- fluency, adequacy, coherence -- that automated metrics cannot reliably capture.7 min→·Intrinsic vs. Extrinsic EvaluationIntrinsic evaluation measures a model component's quality in isolation (e.g., perplexity for a language model), while extrinsic evaluation measures its contribution to a downstream end-task (e.g., translation accuracy).7 min→·NLP for Social GoodNLP technologies can address critical societal challenges -- from extracting life-saving information from clinical notes to preserving endangered languages -- when designed with care for the communities they serve.8 min→·Privacy in NLPLanguage models memorize and can regurgitate sensitive training data -- including personal identifiers, phone numbers, and medical records -- creating privacy risks that require techniques like differential privacy, federated learning, and de-identification to mitigate.9 min→·Responsible NLP DevelopmentResponsible NLP development encompasses the practices, documentation standards, and ethical frameworks -- from model cards to carbon footprint accounting -- that ensure language technologies are built, evaluated, and deployed with transparency, accountability, and awareness of potential harms.9 min→