Editor's Picks
-
Deep Learning's Most Important Ideas - A Brief Historical Review
The goal of this post is to review well-adopted ideas that have stood the test of time. I will present a small set of techniques that cover a lot of basic knowledge necessary to understand modern Deep Learning research. If you're new to the field, these are a great starting point...
-
GPT-3 and A Typology of Hype
Here, I try to deconstruct the buzz about GPT-3, and in trying to do that, I dig deeper into what hype means in the context of emergent technologies and how to integrate the noise out while consuming new science on social media. Read the rest of the post for a framework to think about the buzz in breakthrough technologies while living in the midst of it. GPT-3 or similar models did not assist in any of this writing...
-
Are There Really as Many Neurons in the Human Brain as Stars in the Milky Way?
For a long time, neuroscientists would say that there are about 100 billion neurons in the human brain. Interestingly, no one has ever published a peer-reviewed scientific paper supporting that count. Rather it's been informally interpolated from other measurements. A recent study from 2009 published by Azevedo and colleagues took a crack at a more precise estimate. Their answer is...
A Message From This Week's Sponsor
Data scientists are in demand on Vettery
Vettery is an online hiring marketplace that's changing the way people hire and get hired. Ready for a bold career move? Make a free profile, name your salary, and connect with hiring managers from top employers today.
Data Science Articles & Videos
-
A Tour of End-to-End Machine Learning Platforms
Machine Learning (ML) is known as the high-interest credit card of technical debt. It is relatively easy to get started with a model that is good enough for a particular business problem, but to make that model work in a production environment that scales and can deal with messy, changing data semantics and relationships, and evolving schemas in an automated and reliable fashion, that is another matter altogether. If you’re interested in learning more about a few well-known ML platforms, you’ve come to the right place!...
-
Announcing ScaNN: Efficient Vector Similarity Search
Suppose one wants to search through a large dataset of literary works using queries that require an exact match of title, author, or other easily machine-indexable criteria. Such a task would be well suited for a relational database using a language such as SQL. However, if one wants to support more abstract queries, such as “Civil War poem,” it is no longer possible to rely on naive similarity metrics such as the number of words in common between two phrases...
-
bpe blues +
I might as well talk about how GPT-2/3′s weird tokenizer interacts with arithmetic...Let’s look at how GPT sees numbers from 0 to 9999. (I prepend each numeral with a space because that’s what it will usually see in practice.)...
-
What's new in TensorFlow 2.3?
TensorFlow 2.3 has been released! The focus of this release is on new tools to make it easier for you to load and preprocess data, and to solve input-pipeline bottlenecks, whether you’re working on one machine, or many...
-
Meta-Learning Requires Meta-Augmentation
Meta-learning algorithms aim to learn two components: a model that predicts targets for a task, and a base learner that quickly updates that model when given examples from a new task. This additional level of learning can be powerful, but it also creates another potential source for overfitting, since we can now overfit in either the model or the base learner...We demonstrate that meta-augmentation produces large complementary benefits to recently proposed meta-regularization techniques...
-
Gradient descent for wide two-layer neural networks – II: Generalization and implicit bias
In this blog post, we continue our investigation of gradient flows for wide two-layer “relu” neural networks. In the previous post, Francis explained that under suitable assumptions these dynamics converge to global minimizers of the training objective. Today, we build on this to understand qualitative aspects of the predictor learnt by such neural networks...
-
PyTorch 1.6.0 Release Notes
The PyTorch 1.6 release includes a number of new APIs, tools for performance improvement and profiling, as well as major updates to both distributed data parallel (DDP) and remote procedure call (RPC) based distributed training...
-
PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization
Recent advances in image-based 3D human shape estimation have been driven by the significant improvement in representation power afforded by deep neural networks...[we] formulat[e] a multi-level architecture that is end-to-end trainable. A coarse level observes the whole image at lower resolution and focuses on holistic reasoning. This provides context to an fine level which estimates highly detailed geometry by observing higher-resolution images. We demonstrate that our approach significantly outperforms existing state-of-the-art techniques on single image human shape reconstruction by fully leveraging 1k-resolution input images...
-
Giving GPT-3 a Turing Test
I’ve been playing around with OpenAI’s new GPT-3 language model. When I got beta access, the first thing I wondered was, how human is GPT-3? How close is it to passing a Turing test?...
Training
Quick Question For You: Do you want a Data Science job?
After helping hundred of readers like you get Data Science jobs, we've distilled all the real-world-tested advice into a self-directed course.
The course is broken down into three guides:
- Data Science Getting Started Guide. This guide shows you how to figure out the knowledge gaps that MUST be closed in order for you to become a data scientist quickly and effectively (as well as the ones you can ignore)
- Data Science Project Portfolio Guide. This guide teaches you how to start, structure, and develop your data science portfolio with the right goals and direction so that you are a hiring manager's dream candidate
- Data Science Resume Guide. This guide shows how to make your resume promote your best parts, what to leave out, how to tailor it to each job you want, as well as how to make your cover letter so good it can't be ignored!
Click here to learn more ...
*Sponsored post. If you want to be featured here, or as our main sponsor, contact us!
Jobs
-
Senior Data Scientist - Grubhub - NY / Chicago
Grubhub is looking for a data scientist to join the Pricing team. As a part of Pricing, you’ll be a member of a small team of data scientists and engineers who shape and optimize how we charge our diners, shaping hundreds of millions in revenue annually. You will work closely both with financial stakeholders as well as engineers to ship models that make Grubhub more efficient with the way in which it charges customers. You’ll construct models and A/B tests as well as write code to improve our modeling capabilities...
Want to post a job here? Email us for details >> team@datascienceweekly.org
Training & Resources
-
Video Prediction using Deep Learning and PyTorch (−lightning)
In this guide, I will show you how to code a Convolutional Long Short-Term Memory (ConvLSTM) using an autoencoder (seq2seq) architecture for frame prediction using the MovingMNIST dataset (but custom datasets can also easily be integrated)....
-
Behavioral Testing of NLP models with CheckList
When developing an NLP model, it’s a standard practice to test how well a model generalizes to unseen examples by evaluating it on a held-out dataset...In contrast, the field of software engineering uses a suite of unit tests, integration tests, and end-to-end tests to evaluate all aspects of the product for failures. An application is deployed to production only after passing these rigorous tests...Ribeiro et al. noticed this gap and took inspiration from software engineering to propose an evaluation methodology for NLP called “CheckList”. Their paper won the best overall paper award at ACL 2020...In this post, I will explain the overall concept of CheckList and the various components that it proposes for evaluating NLP models...
Books
-
Seven Databases in Seven Weeks:
A Guide to Modern Databases and the NoSQL Movement
"A book that tries to cover multiple database is a risky endeavor, a book that also provides hands on on each is even riskier but if implemented well leads to a great package. I loved the specific exercises the authors covered. A must read for all big data architects who don’t shy away from coding..."...
For a detailed list of books covering Data Science, Machine Learning, AI and associated programming languages check out our resources page
.
P.S., Enjoy the newsletter? Please forward it to your friends and colleagues - we'd love to have them onboard :) All the best, Hannah & Sebastian