Master Thesis Slides: Topic Development of Methods for Deep Neural Network Architectures Optimization based on Tensor Factorization Algorithms
0 likes6 views
The document presents a research project focused on optimizing deep neural network architectures using tensor factorization algorithms to reduce model size and improve throughput. It outlines objectives such as literature review, experimentation with time series models like N-BEATS, ARIMA, and Transformer, and the development of an optimization method integrated into AutoML solutions. The findings include performance comparisons of the models on the M4 dataset, revealing the efficiency of different approaches to model training and prediction in time series forecasting.
1 of 18
Download to read offline
Ad
Recommended
Deep learning: the final frontier for time series analysis and signal process...
Deep learning: the final frontier for time series analysis and signal process...Alex Honchar - Deep learning, specifically convolutional neural networks and recurrent neural networks, provides alternatives to traditional time series and signal processing techniques that can learn patterns without specifying a model.
- CNNs can learn local geometrical patterns in time series while retaining information over time. RNNs can learn long-range dependencies in high-dimensional data.
- Deep learning approaches like autoencoders are well-suited for tasks like pattern matching, anomaly detection, and simulation/generation of time series. Hybrid approaches that combine hand-crafted features with deep learning may also be effective.
Time Series Forecasting Using Recurrent Neural Network and Vector Autoregress...
Time Series Forecasting Using Recurrent Neural Network and Vector Autoregress...Databricks The document outlines a lecture by Jeffrey Yau on time series forecasting techniques, focusing on both statistical models, like ARIMA and VAR, and machine learning approaches, specifically LSTM networks. It covers problem formulation, implementation in Python, and differences between statistical and neural network methods. The lecture also explores considerations for using Spark in time series forecasting and provides practical steps for building models.
Time series deep learning
Time series deep learningAlberto Arrigoni This document summarizes different methods for time series analysis and prediction in the deep learning era. It discusses classical autoregressive and Bayesian models, general machine learning approaches, and various deep learning techniques including DeepAR, Deep Ensembles, Deep State Space models, and combinations of deep neural networks with Gaussian processes. The document compares the pros and cons of each approach in terms of scalability, ability to share information across time series, handling cold starts with limited data, estimating predictive uncertainty, and dealing with unevenly spaced time series data.
Automated-tuned hyper-parameter deep neural network by using arithmetic optim...
Automated-tuned hyper-parameter deep neural network by using arithmetic optim...IJECEIAES This document discusses the use of the Arithmetic Optimization Algorithm (AOA) for automating the hyper-parameter tuning of deep neural networks (DNNs) particularly in the context of the Lorenz chaotic system. The study demonstrates that AOA outperforms traditional methods like Particle Swarm Optimization (PSO) in terms of accuracy and efficiency. The proposed approach includes a comprehensive framework for optimized hyper-parameter selection, exhibiting improved performance in practical applications compared to manual tuning methods.
Neural Networks and Deep Learning Syllabus
Neural Networks and Deep Learning SyllabusAndres Mendez-Vazquez This document provides an overview of a 65-hour course on neural networks and deep learning taught by Andres Mendez Vazquez at Cinvestav Guadalajara. The course objectives are to introduce students to concepts of neural networks, with a focus on various neural network architectures and their applications. Topics covered include traditional neural networks, deep learning, optimization techniques for training deep models, and specific deep learning architectures like convolutional and recurrent neural networks. The course grades are based on midterms, homework assignments, and a final project.
[DSC Adria 23]Davor Horvatic Human-Centric Explainable AI In Time Series Anal...
[DSC Adria 23]Davor Horvatic Human-Centric Explainable AI In Time Series Anal...DataScienceConferenc1 The document discusses the importance of human-centric explainable AI (HC-XAI) in the context of multivariate time series analysis and forecasting, emphasizing the need for models to offer human-understandable interpretations. It reviews various deep learning architectures and their interpretability, particularly focusing on attention-based models, and outlines benchmarking criteria such as performance, comprehensibility, and user accessibility. Results indicate that while prediction performance is often stable, many models struggle with providing adequate interpretability, highlighting weaknesses in current attention-based approaches.
Using Deep Learning and TensorFlow to Classify Time Series
Using Deep Learning and TensorFlow to Classify Time SeriesAltoros Deep learning techniques like convolutional neural networks (CNNs) show promise for time series classification by automatically learning relevant features from the data. The document discusses applying CNNs to time series classification using TensorFlow, reviewing papers that represent time series as images for CNN input or adapt CNNs to directly handle raw time series. Preliminary tests on an artificial classification problem show CNNs achieving lower error rates than traditional techniques. [/SUMMARY]
Phx dl meetup
Phx dl meetupJames Sirota This document provides information about a Phoenix Deep Learning Meetup Group event. It welcomes attendees and thanks Galvanize for sponsoring. It provides an agenda for the event including a brief introduction to deep learning, use cases, feedforward neural networks, gradient descent, and a TensorFlow example. It invites speakers on topics like scaling TensorFlow and competing libraries. It includes information on who typically attends such events and closes with a quote about AlphaGo defeating a Go champion using deep learning techniques.
deep learning.pptx
deep learning.pptxkhalid aberbach The document is an introduction to deep learning, exploring key concepts such as artificial intelligence, machine learning, and deep learning. It outlines the main types of machine learning—supervised, unsupervised, self-supervised, and reinforcement learning—and discusses fundamental components including data, models, training processes, loss functions, and deep learning frameworks. Notable frameworks mentioned include TensorFlow and PyTorch for building and training neural networks.
ICMS_Tax_Collection_Time_Series.pdf
ICMS_Tax_Collection_Time_Series.pdfGuttenberg Ferreira Passos The document discusses forecasting ICMS tax collection using various statistical and machine learning models, including both linear and non-linear approaches. It highlights the use of time series analysis, deep learning architectures like LSTM, and emphasizes the exploration of historical data to inform decision-making. The results indicate different performance metrics for various models, showcasing the effectiveness of AI in tax forecasting.
Easy to learn deep learning guide - elementry
Easy to learn deep learning guide - elementryAnjaliSohoni The document provides an overview of deep learning, emphasizing its recent advancements and efficacy in various applications like computer vision and natural language processing. It discusses the unique characteristics of deep learning, such as over-parametrization and depth, and introduces key neural network models and training techniques from a statistical perspective. Additionally, the authors explore the theoretical underpinnings of deep learning, aiming to inspire further research in the field.
Multivariate Time Series
Multivariate Time SeriesApache MXNet The document discusses the challenges and advancements in multivariate time series forecasting using autoregressive models, highlighting the limitations of existing methods such as the cold start problem and handling large datasets. It introduces 'DeepAR,' a forecasting model based on autoregressive recurrent neural networks, and outlines its advantages, including minimal manual feature engineering and the ability to forecast with little historical data. Additionally, it presents 'LSTNet,' a model designed to capture both long- and short-term patterns in time series data using a combination of convolutional and recurrent neural networks.
DARMDN: Deep autoregressive mixture density nets for dynamical system mode...
DARMDN: Deep autoregressive mixture density nets for dynamical system mode...Balázs Kégl The document presents research on Deep Autoregressive Mixture Density Nets (DARMN) for modeling dynamics of systems, highlighting its applications in automating engineering tasks and predictive maintenance. It discusses the challenges of integrating AI in engineering, the approach of using generative time-series predictors, and methods for efficient sampling and control. The findings suggest that DARMN exhibits superior performance in model-based reinforcement learning, outperforming previous state-of-the-art techniques in certain scenarios.
System for Prediction of Non Stationary Time Series based on the Wavelet Radi...
System for Prediction of Non Stationary Time Series based on the Wavelet Radi...IJECEIAES This document proposes and examines the performance of a hybrid time series forecasting model called the wavelet radial bases function neural network (WRBFNN) model. The WRBFNN model uses wavelet transforms to extract coefficients from time series data as inputs to a radial basis function neural network for forecasting. The performance of the WRBFNN model is compared to a wavelet feed forward neural network (WFFNN) model using four types of non-stationary time series data. Results show the WRBFNN model achieves more accurate forecasts than the WFFNN model for certain types of non-stationary data, and trains significantly faster with fewer computational resources required.
[20240722_LabSeminar_Huy]WaveForM: Graph Enhanced Wavelet Learning for Long S...
[20240722_LabSeminar_Huy]WaveForM: Graph Enhanced Wavelet Learning for Long S...thanhdowork The document presents 'waveform', a novel framework designed for long sequence forecasting of multivariate time series using discrete wavelet transform (DWT) and graph convolution. It highlights challenges in existing approaches and proposes a global graph to extract interrelationships among data variables while preventing overfitting. Experimental results demonstrate improved predictive performance in tasks involving complex temporal dependencies across multiple datasets.
News recommender system from math model to production solution
News recommender system from math model to production solutionNikolay Karpov This document summarizes a presentation on news recommender systems. It discusses:
1) The speaker's background and experience developing recommender systems for news websites.
2) The challenges of building recommender systems for news domains, which have only implicit feedback and where time is important.
3) The models developed, including matrix factorization models that incorporate time and additional features to address cold start problems for new users and items.
4) How the models were deployed into a full recommendation system using technologies like TensorFlow, Docker, and Flask.
Oleksandr Honchar "Deep learning for signal processing"
Oleksandr Honchar "Deep learning for signal processing"Lviv Startup Club This document discusses using deep learning techniques for signal processing and time series analysis. It begins by describing classical approaches like time domain analysis, frequency domain analysis, and ARMA/ARIMA models. It then covers deep learning techniques like RNNs, CNNs, combinations of RNNs and CNNs, and autoregressive CNNs. It finds that autoregressive CNNs generally perform better than CNNs and RNNs for time series tasks. The document recommends combining deep learning with classical techniques when possible and notes that deep learning has also been successful for tasks like sales forecasting and traffic prediction.
Employing Advanced Neural Networks for Forecasting Time Series Data
Employing Advanced Neural Networks for Forecasting Time Series DataIRJET Journal This paper explores using advanced neural networks like LSTM and GRU for time series forecasting. It tests these techniques on two datasets - one with consistent repeating patterns and one with stock market data that is less predictable. The neural networks were able to accurately forecast future time steps on the repeating pattern data, performing better than a simple baseline method. However, on the less consistent stock market data, the neural networks did not clearly outperform the baseline. The paper concludes these techniques are suitable when time series exhibit repeating patterns but may not capture all relevant information in more unpredictable cases like stock prices. Code for replicating the experiments is made publicly available.
And Then There Are Algorithms
And Then There Are AlgorithmsInfluxData The document discusses the history and definition of algorithms, citing influential figures like Ada Lovelace and Charles Babbage. It explores various types of machine learning algorithms and their applications, including regression, classification, and clustering, providing insights into modern methods such as seq2seq and deep learning techniques. Additionally, it highlights Amazon SageMaker as a platform that facilitates machine learning through built-in algorithms and automated hyperparameter tuning.
C3 w1
C3 w1Ajay Taneja This copyright notice specifies that DeepLearning.AI slides are distributed under a Creative Commons license, can be used non-commercially for education
Build a Neural Network for ITSM with TensorFlow
Build a Neural Network for ITSM with TensorFlowEntrepreneur / Startup This document discusses using machine learning and TensorFlow to build a neural network for IT self-service. It provides an overview of machine learning and neural networks, describing how they are inspired by biological neurons. It also discusses the different types of neural networks and machine learning paradigms like supervised, unsupervised and reinforcement learning. Examples are given of problems neural networks could solve for IT like predicting incident assignment groups or diagnosing issues from chatbots. Popular deep learning frameworks like TensorFlow are also summarized.
Paper sharing_deep learning for smart manufacturing methods and applications
Paper sharing_deep learning for smart manufacturing methods and applicationsYOU SHENG CHEN The document discusses the integration of deep learning techniques in smart manufacturing, highlighting its potential to enhance efficiency and customer satisfaction across various systems. It reviews the evolution of data-driven artificial intelligence, comparing deep learning with traditional methods while detailing its applications in defect prognosis, fault diagnosis, and product quality inspection. Challenges such as data quality and model selection are examined, alongside proposed solutions for effective implementation in manufacturing processes.
Chapter10.pptx
Chapter10.pptxadnansbp The document discusses deep learning techniques. It introduces deep learning and its impact on fields like speech recognition and computer vision. Deep learning uses large datasets and high-capacity models with many parameters to exploit information in data more effectively than traditional machine learning. The document also discusses key developments in deep learning like increased data, deeper network architectures, and accelerated training with GPUs. It provides examples of common neural network architectures, losses, and activation functions used in deep learning.
An Introduction to Neural Architecture Search
An Introduction to Neural Architecture SearchBill Liu The document introduces Neural Architecture Search (NAS), a technique for automating the design of neural networks tailored to specific datasets. It outlines the evolution of deep learning and discusses various optimization methods including Bayesian optimization and reinforcement learning as search strategies. Despite advancements, NAS is not yet widely adopted, facing challenges in reproducibility and consistency compared to human-engineered architectures.
Deep learning for time series pyBCN
Deep learning for time series pyBCNAlex Honchar This document outlines a presentation on deep learning for time series analysis. It introduces the speaker and provides an outline of topics to be covered, including time series in various domains, classical time series analysis approaches, deep learning techniques like RNNs, CNNs and hybrid models, and takeaways. Classical approaches involve time domain, frequency domain and nearest neighbor algorithms as well as ARMA, smoothing and nonlinear dynamic models. Deep learning techniques discussed are RNNs, CNNs, combinations of RNNs and CNNs, and autoregressive CNNs. The presentation concludes with notes on hybrid solutions and takeaways around feature engineering, deep learning surpassing signal processing, and model combinations.
Deep Learning Opening Workshop - Improving Generative Models - Junier Oliva, ...
Deep Learning Opening Workshop - Improving Generative Models - Junier Oliva, ...The Statistical and Applied Mathematical Sciences Institute The document discusses generative models, focusing on change of variable models and autoregressive models, along with their drawbacks related to likelihood and collective modeling. It emphasizes the importance of density estimation in unsupervised learning and explores various generative model architectures like flow models and pixelCNN. The content highlights applications for generating realistic samples in various domains and addresses challenges in modeling dependencies among collections of data.
Dixon Deep Learning
Dixon Deep LearningSciCompIIT This document introduces deep learning approaches for predicting spatio-temporal flows. It discusses how deep learning uses hierarchical layers to model complex nonlinear relationships in spatial and temporal data without assuming a data generation process. Examples are given of applying deep learning to predict traffic flows using loop detector data and to forecast stock price movements using limit order book imbalances. The document outlines the configuration of deep learning models for these tasks and evaluates their performance versus traditional statistical approaches.
Seniors YSC2022
Seniors YSC2022Denis Zakharov Документ обсуждает разработку сервиса, направленного на снижение уровня одиночества среди пенсионеров путем создания сообщества единомышленников и предоставления возможностей для переквалификации. Исследование включает анализ речи для оценки уровня одиночества и разработку цифровой инфраструктуры, которая будет включать веб и мобильное приложения. Основная цель - повысить качество жизни пенсионеров через участие в мероприятиях и развитие технологий психо-социальной поддержки.
Bachelors Diploma Slides Short Version
Bachelors Diploma Slides Short VersionDenis Zakharov Документ представляет исследование применения технологий автоматизации для повышения информационной безопасности в компьютерных сетях, сосредотачиваясь на проблемах прошлого года и актуальных угрозах. Также рассматриваются инструменты автоматизации, процессы в организациях и жизненный цикл программного обеспечения, включая CI/CD и безопасность приложений. В заключение обсуждаются примеры практического применения, такие как использование Ansible и Docker.
More Related Content
Similar to Master Thesis Slides: Topic Development of Methods for Deep Neural Network Architectures Optimization based on Tensor Factorization Algorithms (20)
deep learning.pptx
deep learning.pptxkhalid aberbach The document is an introduction to deep learning, exploring key concepts such as artificial intelligence, machine learning, and deep learning. It outlines the main types of machine learning—supervised, unsupervised, self-supervised, and reinforcement learning—and discusses fundamental components including data, models, training processes, loss functions, and deep learning frameworks. Notable frameworks mentioned include TensorFlow and PyTorch for building and training neural networks.
ICMS_Tax_Collection_Time_Series.pdf
ICMS_Tax_Collection_Time_Series.pdfGuttenberg Ferreira Passos The document discusses forecasting ICMS tax collection using various statistical and machine learning models, including both linear and non-linear approaches. It highlights the use of time series analysis, deep learning architectures like LSTM, and emphasizes the exploration of historical data to inform decision-making. The results indicate different performance metrics for various models, showcasing the effectiveness of AI in tax forecasting.
Easy to learn deep learning guide - elementry
Easy to learn deep learning guide - elementryAnjaliSohoni The document provides an overview of deep learning, emphasizing its recent advancements and efficacy in various applications like computer vision and natural language processing. It discusses the unique characteristics of deep learning, such as over-parametrization and depth, and introduces key neural network models and training techniques from a statistical perspective. Additionally, the authors explore the theoretical underpinnings of deep learning, aiming to inspire further research in the field.
Multivariate Time Series
Multivariate Time SeriesApache MXNet The document discusses the challenges and advancements in multivariate time series forecasting using autoregressive models, highlighting the limitations of existing methods such as the cold start problem and handling large datasets. It introduces 'DeepAR,' a forecasting model based on autoregressive recurrent neural networks, and outlines its advantages, including minimal manual feature engineering and the ability to forecast with little historical data. Additionally, it presents 'LSTNet,' a model designed to capture both long- and short-term patterns in time series data using a combination of convolutional and recurrent neural networks.
DARMDN: Deep autoregressive mixture density nets for dynamical system mode...
DARMDN: Deep autoregressive mixture density nets for dynamical system mode...Balázs Kégl The document presents research on Deep Autoregressive Mixture Density Nets (DARMN) for modeling dynamics of systems, highlighting its applications in automating engineering tasks and predictive maintenance. It discusses the challenges of integrating AI in engineering, the approach of using generative time-series predictors, and methods for efficient sampling and control. The findings suggest that DARMN exhibits superior performance in model-based reinforcement learning, outperforming previous state-of-the-art techniques in certain scenarios.
System for Prediction of Non Stationary Time Series based on the Wavelet Radi...
System for Prediction of Non Stationary Time Series based on the Wavelet Radi...IJECEIAES This document proposes and examines the performance of a hybrid time series forecasting model called the wavelet radial bases function neural network (WRBFNN) model. The WRBFNN model uses wavelet transforms to extract coefficients from time series data as inputs to a radial basis function neural network for forecasting. The performance of the WRBFNN model is compared to a wavelet feed forward neural network (WFFNN) model using four types of non-stationary time series data. Results show the WRBFNN model achieves more accurate forecasts than the WFFNN model for certain types of non-stationary data, and trains significantly faster with fewer computational resources required.
[20240722_LabSeminar_Huy]WaveForM: Graph Enhanced Wavelet Learning for Long S...
[20240722_LabSeminar_Huy]WaveForM: Graph Enhanced Wavelet Learning for Long S...thanhdowork The document presents 'waveform', a novel framework designed for long sequence forecasting of multivariate time series using discrete wavelet transform (DWT) and graph convolution. It highlights challenges in existing approaches and proposes a global graph to extract interrelationships among data variables while preventing overfitting. Experimental results demonstrate improved predictive performance in tasks involving complex temporal dependencies across multiple datasets.
News recommender system from math model to production solution
News recommender system from math model to production solutionNikolay Karpov This document summarizes a presentation on news recommender systems. It discusses:
1) The speaker's background and experience developing recommender systems for news websites.
2) The challenges of building recommender systems for news domains, which have only implicit feedback and where time is important.
3) The models developed, including matrix factorization models that incorporate time and additional features to address cold start problems for new users and items.
4) How the models were deployed into a full recommendation system using technologies like TensorFlow, Docker, and Flask.
Oleksandr Honchar "Deep learning for signal processing"
Oleksandr Honchar "Deep learning for signal processing"Lviv Startup Club This document discusses using deep learning techniques for signal processing and time series analysis. It begins by describing classical approaches like time domain analysis, frequency domain analysis, and ARMA/ARIMA models. It then covers deep learning techniques like RNNs, CNNs, combinations of RNNs and CNNs, and autoregressive CNNs. It finds that autoregressive CNNs generally perform better than CNNs and RNNs for time series tasks. The document recommends combining deep learning with classical techniques when possible and notes that deep learning has also been successful for tasks like sales forecasting and traffic prediction.
Employing Advanced Neural Networks for Forecasting Time Series Data
Employing Advanced Neural Networks for Forecasting Time Series DataIRJET Journal This paper explores using advanced neural networks like LSTM and GRU for time series forecasting. It tests these techniques on two datasets - one with consistent repeating patterns and one with stock market data that is less predictable. The neural networks were able to accurately forecast future time steps on the repeating pattern data, performing better than a simple baseline method. However, on the less consistent stock market data, the neural networks did not clearly outperform the baseline. The paper concludes these techniques are suitable when time series exhibit repeating patterns but may not capture all relevant information in more unpredictable cases like stock prices. Code for replicating the experiments is made publicly available.
And Then There Are Algorithms
And Then There Are AlgorithmsInfluxData The document discusses the history and definition of algorithms, citing influential figures like Ada Lovelace and Charles Babbage. It explores various types of machine learning algorithms and their applications, including regression, classification, and clustering, providing insights into modern methods such as seq2seq and deep learning techniques. Additionally, it highlights Amazon SageMaker as a platform that facilitates machine learning through built-in algorithms and automated hyperparameter tuning.
C3 w1
C3 w1Ajay Taneja This copyright notice specifies that DeepLearning.AI slides are distributed under a Creative Commons license, can be used non-commercially for education
Build a Neural Network for ITSM with TensorFlow
Build a Neural Network for ITSM with TensorFlowEntrepreneur / Startup This document discusses using machine learning and TensorFlow to build a neural network for IT self-service. It provides an overview of machine learning and neural networks, describing how they are inspired by biological neurons. It also discusses the different types of neural networks and machine learning paradigms like supervised, unsupervised and reinforcement learning. Examples are given of problems neural networks could solve for IT like predicting incident assignment groups or diagnosing issues from chatbots. Popular deep learning frameworks like TensorFlow are also summarized.
Paper sharing_deep learning for smart manufacturing methods and applications
Paper sharing_deep learning for smart manufacturing methods and applicationsYOU SHENG CHEN The document discusses the integration of deep learning techniques in smart manufacturing, highlighting its potential to enhance efficiency and customer satisfaction across various systems. It reviews the evolution of data-driven artificial intelligence, comparing deep learning with traditional methods while detailing its applications in defect prognosis, fault diagnosis, and product quality inspection. Challenges such as data quality and model selection are examined, alongside proposed solutions for effective implementation in manufacturing processes.
Chapter10.pptx
Chapter10.pptxadnansbp The document discusses deep learning techniques. It introduces deep learning and its impact on fields like speech recognition and computer vision. Deep learning uses large datasets and high-capacity models with many parameters to exploit information in data more effectively than traditional machine learning. The document also discusses key developments in deep learning like increased data, deeper network architectures, and accelerated training with GPUs. It provides examples of common neural network architectures, losses, and activation functions used in deep learning.
An Introduction to Neural Architecture Search
An Introduction to Neural Architecture SearchBill Liu The document introduces Neural Architecture Search (NAS), a technique for automating the design of neural networks tailored to specific datasets. It outlines the evolution of deep learning and discusses various optimization methods including Bayesian optimization and reinforcement learning as search strategies. Despite advancements, NAS is not yet widely adopted, facing challenges in reproducibility and consistency compared to human-engineered architectures.
Deep learning for time series pyBCN
Deep learning for time series pyBCNAlex Honchar This document outlines a presentation on deep learning for time series analysis. It introduces the speaker and provides an outline of topics to be covered, including time series in various domains, classical time series analysis approaches, deep learning techniques like RNNs, CNNs and hybrid models, and takeaways. Classical approaches involve time domain, frequency domain and nearest neighbor algorithms as well as ARMA, smoothing and nonlinear dynamic models. Deep learning techniques discussed are RNNs, CNNs, combinations of RNNs and CNNs, and autoregressive CNNs. The presentation concludes with notes on hybrid solutions and takeaways around feature engineering, deep learning surpassing signal processing, and model combinations.
Deep Learning Opening Workshop - Improving Generative Models - Junier Oliva, ...
Deep Learning Opening Workshop - Improving Generative Models - Junier Oliva, ...The Statistical and Applied Mathematical Sciences Institute The document discusses generative models, focusing on change of variable models and autoregressive models, along with their drawbacks related to likelihood and collective modeling. It emphasizes the importance of density estimation in unsupervised learning and explores various generative model architectures like flow models and pixelCNN. The content highlights applications for generating realistic samples in various domains and addresses challenges in modeling dependencies among collections of data.
Dixon Deep Learning
Dixon Deep LearningSciCompIIT This document introduces deep learning approaches for predicting spatio-temporal flows. It discusses how deep learning uses hierarchical layers to model complex nonlinear relationships in spatial and temporal data without assuming a data generation process. Examples are given of applying deep learning to predict traffic flows using loop detector data and to forecast stock price movements using limit order book imbalances. The document outlines the configuration of deep learning models for these tasks and evaluates their performance versus traditional statistical approaches.
Deep Learning Opening Workshop - Improving Generative Models - Junier Oliva, ...
Deep Learning Opening Workshop - Improving Generative Models - Junier Oliva, ...The Statistical and Applied Mathematical Sciences Institute
More from Denis Zakharov (9)
Seniors YSC2022
Seniors YSC2022Denis Zakharov Документ обсуждает разработку сервиса, направленного на снижение уровня одиночества среди пенсионеров путем создания сообщества единомышленников и предоставления возможностей для переквалификации. Исследование включает анализ речи для оценки уровня одиночества и разработку цифровой инфраструктуры, которая будет включать веб и мобильное приложения. Основная цель - повысить качество жизни пенсионеров через участие в мероприятиях и развитие технологий психо-социальной поддержки.
Bachelors Diploma Slides Short Version
Bachelors Diploma Slides Short VersionDenis Zakharov Документ представляет исследование применения технологий автоматизации для повышения информационной безопасности в компьютерных сетях, сосредотачиваясь на проблемах прошлого года и актуальных угрозах. Также рассматриваются инструменты автоматизации, процессы в организациях и жизненный цикл программного обеспечения, включая CI/CD и безопасность приложений. В заключение обсуждаются примеры практического применения, такие как использование Ansible и Docker.
Student Spring 2022
Student Spring 2022Denis Zakharov This document discusses information security in automating vSphere infrastructure deployment using PowerCLI. It describes how PowerCLI can be used to dynamically configure application services, reduce errors from human errors, and reduce operating costs through automation. The document provides a script example to deploy virtual machines from a template to multiple folders using a for loop. It also discusses adding extra features like changing network adapter settings and adding a progress bar to monitor task status.
Development of a plugin for VS Code that supports ACSL language.
Development of a plugin for VS Code that supports ACSL language.Denis Zakharov Документ описывает разработку плагина для VS Code, поддерживающего язык ACSL, с целью улучшения опыта разработчиков и знакомства с документацией. В нем рассматриваются этапы разработки, включая планирование, тестирование и мониторинг, а также используемые технологии и модули. Кроме того, предоставлены ключевые слова и структура проекта, включая конфигурацию и описание используемых файлов.
Exploring ML methods to increase the profitability of the trading strategy
Exploring ML methods to increase the profitability of the trading strategyDenis Zakharov The document discusses various machine learning methods used to enhance trading strategy profitability, focusing on ensuring time series data is stationary before applying statistical models. It outlines data preparation techniques such as Box-Cox transformation, differentiation, and fractional differentiation, as well as different regression methods and their performance metrics. The author, Denis Zakharov, references various libraries and tools while hinting at future applications, including stock predictions.
Student Spring 2020
Student Spring 2020Denis Zakharov The document discusses delta modulation, a signal encoding technique. Delta modulation has several advantages, including simplicity of encoding and decoding where the original signal can be recovered by integrating the encoded signal. It also has benefits like reduced susceptibility to interference due to the separation of harmonic sidebands in frequency. Delta modulation additionally improves the signal-to-noise ratio for low-level signals through companding.
Student Spring 2019
Student Spring 2019Denis Zakharov Документ обсуждает разнообразные аспекты информационной безопасности, включая исторические попытки расшифровки шифров, такие как 'Энигма', и современные угрозы, такие как мобильные трояны и ransomware. Он упоминает недостатки защиты от взлома, особенно в играх с использованием технологии Denuvo, и приводит статистику по вредоносным программам и атакам на пользователей в 2018 году. Документ также содержит практические советы от Эдварда Сноудена по обеспечению личной безопасности в интернете.
Student Spring 2021
Student Spring 2021Denis Zakharov The document summarizes the student's research on approaches to secure software development. It outlines various stages of a project pipeline and software development lifecycles. It also discusses common security issues like SQL injection, buffer overflows, and weaknesses in cryptography. The research aims to propose a methodology that incorporates security features and reviews at each stage of the development process. The student references materials on software security best practices and smart contract vulnerabilities to inform the research.
DocHolder
DocHolderDenis Zakharov Документ описывает сервис хостинга файлов 'docholder', разработанный студентами, который предоставляет функционал для управления компаниями и документами. Включает авторизацию пользователей, навигацию по компаниям и систему оповещений, а также базируется на таких технологиях, как Vue.js и Spring. Портал ориентирован на удобство использования и обладает будущими возможностями, такими как чат и автозаполнение.
Ad
Recently uploaded (20)
Single-Cell Multi-Omics in Neurodegeneration p1.pptx
Single-Cell Multi-Omics in Neurodegeneration p1.pptxKanakChaudhary10 Decoding Brain Complexity at Single-Cell Resolution
Single-Cell Transcriptomics in Neurodegeneration
Epigenomics and Gene Regulation
Proteomics at Single-Cell Resolution
Metabolomics in Cellular Context
Novel Therapeutic Targets
Cell-Type-Specific Mechanisms
How Psychology Can Power Product Decisions: A Human-Centered Blueprint- Shray...
How Psychology Can Power Product Decisions: A Human-Centered Blueprint- Shray...ShrayasiRoy2 In an era where users are bombarded with endless choices, capturing attention and driving meaningful engagement isn't just about building features — it's about understanding what drives human behavior at its core. This presentation offers a deep dive into how psychological principles can inform smarter, more intuitive product decisions. It’s not just theory — it’s a hands-on blueprint for applying human-centered thinking at every stage of product development.
Grounded in behavioral science, consumer psychology, and cognitive design, this deck unpacks the key psychological drivers behind user motivation, decision-making, emotional engagement, and habit formation. From attention economics and dopamine-driven interactions to trust cues, loss aversion, and the paradox of choice — you'll see how to harness what the mind naturally does to build digital products people don’t just use… they return to, talk about, and even advocate for.
We’ll explore:
Cognitive biases that shape user perception and choices — and how to use them to your advantage without crossing ethical lines.
Emotional design principles that build trust, trigger desire, and turn features into feelings.
User behavior loops that build stickiness, deepen retention, and create emotional investments.
Psychological friction — when to reduce it for conversion, and when to add it for intentionality.
Dark patterns vs. ethical persuasion — the thin line between influence and manipulation.
A/B testing with a psychological lens — learning not just what works, but why it works.
This is more than UX research or product marketing fluff. It’s a call to build with empathy, backed by science, and sharpened by real-world product thinking. Whether you’re shaping onboarding flows, gamifying engagement, designing pricing models, or rethinking retention strategies — this deck will arm you with a new lens: the human mind.
Perfect for product managers, growth strategists, UX designers, behavioral scientists, and anyone serious about building products that resonate, retain, and inspire action.
Because at the heart of every great product… is a human.
Let’s start building for the brain — not just the screen.
GBSN__Unit 2 - Control of Microorganisms
GBSN__Unit 2 - Control of MicroorganismsAreesha Ahmad Microbiology for Nursing students - According to New PNC course curriculum - 2025
Instrumentation of IR and Raman Spectrophotometers.pptx
Instrumentation of IR and Raman Spectrophotometers.pptxsngth2h2acc Instrumentation of IR and Raman Spectrophotometers
The scientific heritage No 162 (162) (2025)
The scientific heritage No 162 (162) (2025)The scientific heritage The scientific heritage No 162 (162) (2025)
EV REGENERATIVE ACCELERATION INNOVATION SUMMARY PITCH June 13, 2025.pdf
EV REGENERATIVE ACCELERATION INNOVATION SUMMARY PITCH June 13, 2025.pdfThane Heins NOBEL PRIZE WINNING ENERGY RESEARCHER Summary: This document presents a breakthrough in electric vehicle (EV) innovation— the EV Regenerative Acceleration Innovation (EVRAI)—developed and patented by Thane Heins Energy Inc. of Ottawa, Canada, Peer-reviewed, validated and independently replicated at the University of Ottawa following performance confirmations at MIT. EVRAI now provides all EVs and hybrid vehicles with the ability to completely recharge themselves as they drive, eliminating the need for external plug-in recharging and reducing battery size requirements by up to 80%. This document outlines the new physics principles, internationally confirmed, validated replications, individual and commercial implications of this paradigm-shifting innovation in the context of sustainable mobility.
The Gender Binary & LGBTI People: Religious Myth and Medical Malpractice
The Gender Binary & LGBTI People: Religious Myth and Medical MalpracticeVeronica Drantz, PhD Designated as disordered rather than just different, LGBTI people have been and continue to be victims of medical malpractice purely because they are neither Adams nor Eves. Psychiatrists, surgeons, endocrinologists, pediatricians, and other medical experts have subjected LGBTI people to bogus and horrific treatments with reckless disregard for patient health and well-being―all the while ignoring the basic tenets of medical ethics and the ever-growing scientific evidence showing LGBTI people to be natural variations. Beyond this, medical stigmatization of LGBTI people has contributed to their oppression in the world at large. This treatise will contrast the scientific evidence with the ongoing medical (mis)treatment of LGBTI people to vividly illustrate the insidious effect of the biblical creation myth.
Investigatory_project Topic:-effect of electrolysis in solar desalination .pdf
Investigatory_project Topic:-effect of electrolysis in solar desalination .pdfshubham997ku Cbse class 12. chemistry investigatory project
Herbal Excipients: Natural Colorants & Perfumery Agents
Herbal Excipients: Natural Colorants & Perfumery AgentsSeacom Skills University Excipients can be defined as non active ingredients that are mixed with therapeutically active compounds to form medicines. The excipients are the substance which are used as a medium for giving a medicament.These help in processing of the drug delivery system during its manufacture, protect, support or enhance stability, bioavailability or patient acceptability.
The word comes from the Latin word meaning a sweets melling fluid contain the essence of flowers and other substances. Perfume is a mixture of fragrant essential oil and aroma compounds Fragrances used for external applications such as spray perfumes, body care, homecare, cosmetics, soaps and detergents.
MEDICINAL CHEMISTRY PROSPECTIVES IN DESIGN OF EGFR INHIBITORS.pptx
MEDICINAL CHEMISTRY PROSPECTIVES IN DESIGN OF EGFR INHIBITORS.pptx40RevathiP Introduction , importance and role of EGFR and EGFR inhibitors
Study of Appropriate Information Combination in Image-based Obfuscated Malwar...
Study of Appropriate Information Combination in Image-based Obfuscated Malwar...takahashi34 This slide introduces our study titled "Study of Appropriate Information Combination in Image-based Obfuscated Malware Detection", which was presented at DSN2025.
To tackle the threat of obfuscated malware, our study proposes an image-based detection method that combines four types of information—byte values, entropy, semantic information, and bigrams frequency—from Windows PE executable files. Proposed method is evaluated in a realistic environment containing both obfuscated and non-obfuscated benign and malicious files. Evaluation results show that combining these information types achieves 88.66% accuracy, demonstrating the effectiveness of information combination for detecting obfuscated malware.
GBSN_Unit 3 - Medical and surgical Asepsis
GBSN_Unit 3 - Medical and surgical AsepsisAreesha Ahmad Microbiology for Nursing students - According to New PNC course curriculum - 2025
GBSN_ Unit 1 - Introduction to Microbiology
GBSN_ Unit 1 - Introduction to MicrobiologyAreesha Ahmad Microbiology for Nursing students - According to New PNC course curriculum - 2025
It's about the habitat of organisms. Where they live and what they need to su...
It's about the habitat of organisms. Where they live and what they need to su...KwabenaAbrokwah1 It's about the habitat of organisms. Where they live and what they need to survive.
Science grade 7 assesement Quarter I based on matatag curriculum
Science grade 7 assesement Quarter I based on matatag curriculumBryanLebasnon1 30 items assesment for matatag curriculum in grade 7 science
Relazione di laboratorio Idrolisi dell'amido (in inglese)
Relazione di laboratorio Idrolisi dell'amido (in inglese)paolofvesco relazione di laboratorio sull'idrolisi dell'amido
Overview of Stem Cells and Immune Modulation.ppsx
Overview of Stem Cells and Immune Modulation.ppsxAhmedAtwa29 This presentation, "Stem Cells & Immune Modulation: The Future of Regenerative Medicine?" by Dr. Ahmed Atwa, explores the groundbreaking potential of stem cells in regenerative medicine and immune regulation. It covers stem cell types (Embryonic, Adult, iPSCs), their unique properties (self-renewal, differentiation), and therapeutic applications in tissue repair, disease modeling, and immunomodulation. Special focus is given to mesenchymal stem cells (MSCs) and their role in modulating immune responses through cytokines, extracellular vesicles (EVs), and preconditioning strategies. The slides also address clinical challenges, cancer stem cell evasion, and future directions in stem cell therapy.
EV REGENERATIVE ACCELERATION INNOVATION SUMMARY PITCH June 13, 2025.pdf
EV REGENERATIVE ACCELERATION INNOVATION SUMMARY PITCH June 13, 2025.pdfThane Heins NOBEL PRIZE WINNING ENERGY RESEARCHER
Ad
Master Thesis Slides: Topic Development of Methods for Deep Neural Network Architectures Optimization based on Tensor Factorization Algorithms
- 1. Development of Methods for Deep Neural Network Architectures Optimization based on Tensor Factorization Algorithms Supervisor: Revin Ilya Evgenyevich, research associate, Laboratory for Composite AI, Research Center “Strong AI in Industry” Presented by: Zakharov Denis, J4232c
- 2. Pain: Current deep networks have excessive parameters which results in long training time and expensive storing wights production. Hypothesis: With the use of optimization method we can reduce size of stored model, increase throughput. Work: Development of Methods for Deep Neural Network Architectures Optimization based on Tensor Factorization Algorithms Problem 2
- 3. Purpose and objectives of study Goal Development of Methods for Deep Neural Network Architectures Optimization based on Tensor Factorization Algorithms Objectives • Make a Literature review of related field to provide a background • Make a scientific research of tensor Algorithms (Optimization, Operations) • Make a scientific research of LoRA approach • Perform experiment with TS Models • Based on these findings propose a product that can be integrated into AutoML solution • Develop Optimization Method • Contribute to Fedot.Industrial framework 3
- 4. BERT 340M T5 11B GPT-3 175B Megatron-Turing 530B GPT-4 1.76T Gemini Pro ≈30T Gemini Ultra ≈60T 2018 2019 2020 2021 2022 2023 2024 Model Trend Modern Networks 4 Number of parameters
- 5. Tensor Decomposition 5 ! ≈ = !! "! #! " ! "#$ $ % & ! ≈ !! !" !# !! !" !# ! !! ≈ "" #! $! ! ! "#$ ! " # $! = ∗ ∗ t-SVD (Singular Value Decomposition) CANDECOMP/PARAFAC Tucker Block Term
- 6. Low-rank decomposition 6 𝑀 𝑅! " 𝐿! ≈ × 𝑚×𝑘 𝑚×𝑛 𝑘×𝑛
- 7. Statement of Experiment 7 Check performance of TS models NBEATS, Transformer, ARIMA on M4 dataset for checking how would they predict timeseries based on data.
- 8. Models 8 Transformer — state-of-the-art deep learning model introduced in 2017. It is an encoder-decoder architecture whose core feature is the ‘multi-head attention’ mechanism Linear Linear Linear Scaled Dot-Product Attention Concat Linear Multi-Head Attention MatMul SoftMax Mask Scale MatMul Scaled Dot-Product Attention In Out Multi-Head Attention Add & Norm Feed Forward Add & Norm Masked Multi-Head Attention Add & Norm Multi-Head Attention Add & Norm Feed Forward Add & Norm SoftMax Linear ! × ! ×
- 9. Models 9 NBEATS — deep learning-based approach for time series forecasting. ARIMA — popular statistical model used to forecast future values in a time series based on past values. Block FC Stack FC FC !!(#!) !"(#") Backcast Stack Forecast Stack Block 1 Block 2 Block K … Forecast Global Forecast Stack 1 Stack 2 Stack M …
- 10. M4 Dataset 10 The M4 dataset is a collection of 100,000 time series used for the fourth edition of the Makridakis forecasting Competition. Consists of time series of: • Yearly — 63 avg training length • Quarterly — 125 • Monthly — 302 • Weekly — 2035 • Daily — 475 • Hourly — 682
- 11. Experiment - Monthly 11 ARIMA NBEATS Transformer
- 12. Interpretation of results 12 On short series all three models performed well and suitable for predicting However with longer range • NBEATS perform better than others models • Some models has a critical difference in predictions • Longer Range all models not struggle Training on just 50 epochs took: • Almost an hour for NBEATS • 20 min for Transformer • 1 min for Arima
- 13. LoRA + rSVD ! ∈ #!×! A = "(0, &! ) B = 0 r x h !! "×$ % $×& ' "×& = ( "×" ) "×" *! "×& SVD recover ( $×" = ( + "×" ! $×" 13
- 14. Results NBEATS Base Model LoRA Layers Model latency throughput No LoRA 0.00108 4364808.0 LoRA Layer 0.00166 4159606.0 MS Default 0.00106 4748564.0 MS All 0.00108 4482670.0 MS LoRA 0.00105 5365210.0 14
- 15. Results Transformer Model latency throughput No LoRA 0.00085 9306328.0 LoRA Layer 0.00084 9782205.0 MS Default 0.00085 9572622.0 MS All 0.00085 9463282.0 MS LoRA 0.00084 8586088.0 0 0,01 0,02 0,03 0,04 0,05 0,06 0,07 Epoch 1 Epoch 2 Epoch 3 Epoch 4 Epoch 5 Epoch 6 Epoch 7 Epoch 8 Model Train With Early Stopping Base LoRA 15
- 17. Resume In this work: • Provide an Experiment for research of performance: • NBEATS • ARIMA • Transformer • Logically assumed that layers in models could be replaced using LoRa approach • Implement logic as a master thesis 15 TN diagrams of some popular decompositions !(") !($) !(%) "(") "($) "(%) "(&) !! !" !# !$ !(",") !(%,") !(&,") !(",%) !(%,%) !(&,%) !(",&) !(%,&) !(&,&) "" "% "& "' "( ") "* "+ ", !(") !" !# !$ … !($) !(%) %" %%&" %$ !(") !" !# !$ … !($) !(%) %" %%&" %$ %' !(") "$ "" "% … !(%) !($) !! !" !# # !(") "$ "" "% … !(%) !($) # !! !" !# !(") "$ "" "% … !(%) !($) # !! !" !# #(")
- 18. THANK YOU FOR YOUR TIME! @misterzurg