[ML News] DeepMind builds Gopher | Google builds GLaM | Suicide capsule uses AI to check access

#mlnews #gopher #glam

Your updates on everything going on in the Machine Learning world.

Sponsor: Weights & Biases https://wandb.me/yannic

OUTLINE: 0:00 - Intro & Overview 0:20 - Sponsor: Weights & Biases 3:05 - DeepMind releases 3 papers on large language models 11:45 - Hugging Face Blog: Training CodeParrot from scratch 14:25 - Paper: Pre-Training vision systems with noise 15:45 - DeepMind advances Quantum Mechanics 16:45 - GoogleAI trains GLaM: 1 Trillion Parameters Mixture of Experts Model 18:45 - Colin Raffel calls for building ML models like we build Open-Source software 22:05 - A rebuke of the hype around DeepMind's math paper 24:45 - Helpful Things 32:25 - Suicide Capsule plans AI to assess your mental state before use 35:15 - Synthesia raises 50M to develop AI avatars

Weights & Biases Embedding Projector https://twitter.com/_ScottCondron/status/1469411468139536385?utm_source=pocket_mylist https://docs.wandb.ai/ref/app/features/panels/weave/embedding-projector https://wandb.ai/timssweeney/toy_datasets/reports/Feature-Report-W-B-Embeddings-Projector--VmlldzoxMjg2MjY4?accessToken=bo36zrgl0gref1th5nj59nrft9rc4r71s53zr2qvqlz68jwn8d8yyjdz73cqfyhq

DeepMind releases 3 papers on large language models https://deepmind.com/blog/article/language-modelling-at-scale https://arxiv.org/pdf/2112.04426.pdf https://kstatic.googleusercontent.com/files/b068c6c0e64d6f933068f7de30ea722359ef87c6c14d3065856b86d44fbdf2dea3ff373ed9eb751514f242d20df9d6a468622fad093f962563545e7d0cdb9dba https://arxiv.org/pdf/2112.04359.pdf https://deepmind.com/research/publications/2021/improving-language-models-by-retrieving-from-trillions-of-tokens

Hugging Face Blog: Training CodeParrot from scratch https://huggingface.co/blog/codeparrot?utm_source=pocket_mylist

Paper: Pre-Training vision systems with noise https://mbaradad.github.io/learning_with_noise/

DeepMind advances Quantum Mechanics https://deepmind.com/blog/article/Simulating-matter-on-the-quantum-scale-with-AI https://storage.googleapis.com/deepmind-media/papers/Data_Driven_Density_Functional_Design/data_driven_density_functional_design_unformatted.pdf https://github.com/deepmind/deepmind-research/tree/master/density_functional_approximation_dm21

GoogleAI trains GLaM: 1 Trillion Parameters Mixture of Experts Model https://ai.googleblog.com/2021/12/more-efficient-in-context-learning-with.html

Colin Raffel calls for building ML models like we build Open-Source software https://colinraffel.com/blog/a-call-to-build-models-like-we-build-open-source-software.html

A rebuke of the hype around DeepMind's math paper https://arxiv.org/abs/2112.04324?s=09

Helpful Things https://twitter.com/huggingface/status/1468996110207401992 https://docs.cohere.ai/prompt-engineering-wiki/?utm_source=pocket_mylist https://github.blog/2021- ... https://www.youtube.com/watch?v=ZOkvFf8JbkA