Network Architectures - Part 2
-
- 0 Rating
- 0 Reviews
- 0 Students Enrolled
Network Architectures - Part 2
RNNs for sequential data are positioned relative to state-space models and trained as probabilistic language models via cross-entropy with truncated backpropagation and regularization. Gated architectures—LSTM and GRU—address long-range dependencies, while newer variants and hybrid state-space/RNN approaches push efficiency and context length beyond classic gated designs.
-
- 0 Rating
- 0 Reviews
- 0 Students Enrolled
Requirements
- see DELTA course requirements
General Overview
Description
Network Architectures for Sequential Data – Recurrent Neural Networks II revisits language models (LMs), contrasts state-space models with RNNs, and clarifies how recurrent hidden states encode evolving context. RNN language model training is framed as next-token prediction with cross-entropy, typically using teacher forcing and truncated backpropagation through time, with attention to vanishing/exploding gradients, gradient clipping, dropout, and weight tying. Gated RNNs for the rescue explains how LSTM and GRU use gates to preserve and update information over long ranges, including common extensions such as peephole connections, layer normalization, bidirectionality, and stacked depths. Beyond gated recurrent networks offers a glance at cutting-edge designs—attention-augmented RNNs, SRU/QRNN, dilated and residualized RNNs, RWKV, and modern state-space sequence models—that trade off accuracy, speed, and sequence length handling.
Courses that include this CL
Recommended for you
Meet the instructors !
Stefan received a PhD from the University of Hamburg in 2007, where he also completed his habilitation on decision analysis and support using ensemble forecasting models in 2012. He then joined the Humboldt-University of Berlin in 2014, where he heads the Chair of Information Systems at the School of Business and Economics. He serves as an associate editor for the International Journal of Business Analytics, Digital Finance, and the International Journal of Forecasting, and as department editor of Business and Information System Engineering (BISE). Stefan has secured substantial amounts of research funding and published several papers in leading international journals and conferences. His research concerns the support of managerial decision-making using quantitative empirical methods. He specializes in applications of (deep) machine learning techniques in the broad scope of marketing and risk analytics. Stefan actively participates in knowledge transfer and consulting projects with industry partners; from start-up companies to global players and not-for-profit organizations.