Dominik Farhan

How to: Magpie (Doing well in Mr. Straka's courses)

This post is inspired by a great article by Andrej Karpathy.

This is a guide. Not a manual. Not a cheatsheet. And it is biased. It is also a story of survival, a little bit of a cookbook for grinding points, and a letter of admiration to all great teachers.

But let's start at the beginning. October is here and I am starting my first year as an undergrad. And for some unrelated reasons, all I want is peace, get those thirty credits and enjoy some student life. Yet, here I am, in N1, watching this man jump around while he is erratically moving his hands and legs. I'm hooked. Hello, machine learning. Bye-bye life.

Masterpiece by Ondra Sladky

Is it for you?

These courses are time demanding. I don't think that I would have ever finished them if it wasn't for friends who shared the burden of debugging a lot of code with me. However, it is also possible to solve excercises alone. There are students who do it. But we will get to it.

During the semester when I did Deep Learning it sometimes felt like I was studying at two different schools. One was MFF the second was Straka's course. It is hard to estimate but I often spent like 4-hours a week doing the assignments for Greenhorns and one of my friends suggested that the difficulty of doing them is a little higher than doing C# by Mr. Ježek (multiply the difficulty by a factor of 22 if you think about the Deep Learning course). Also note that the given hours are from someone who was most of the time teaming up with two other people, so for solo-person they ought to be much higher.

I don't want to discourage you but it is true that it takes time and you should do most of the assignments. Solving problems is the backbone of these courses and in my opinion the best way to study.

Theoretically, it is possible to pass those courses “just” by doing two-thirds of the assignments but I always felt like I learn more by doing the majority of assignments and also those additional points are pure gold when it comes to the exam. My style was to work a lot during the semester and take it easy on the test.

To also talk about the good things, you can get some great value. You are garanteed to get a good overview of ML and Deep Learning. You definitely won't be ready to create, train, and deploy huge ML models but you will have a good knowledge of where to start when you need one. The courses should give you a massive toolbox of ML algorithms that you ought to perfect by yourself after finishing. It is also a high-quality course. What I mean by that is that even if you didn't like the theme or thought that something might be presented in a more digestible way, you would marvel how much time Milan Straka must have spent creating this AI-monster-trilogy.

I also can't help myself from not mentioning it but I think Milan Straka is an innovative teacher. He likes to try new things. For example, during my Deep Learning semester, he decided to give consultations prior to each exam. If you needed, you could come an hour or two earlier and discuss whatever you weren't sure about. This actually saved my exam because both questions I asked him about ended up being in my test.

I've been interested in AI for a long time and enjoyed both ML and DL even when I took it in the first year. However, I don't think it's for everybody. Nonetheless, there is a simple algorithm to determine if those courses are good for you. It can be summed up with the words of Bob Iger who put it very simply,

If something doesn't feel right to you, then it's probably not right to you.

Anyway, you can just try it and remove it from your timetable in the first few weeks.

Getting started

Get a team

You don't need to do this right at the beginning. You should probably solve at least the first two or three lecture assignments alone to get a feel for it. But later, having a team, always helped me.

Often my friends could come up with better model architecture or improve mine. Sometimes we were doing one assignment for several hours, sitting waxed to one screen and debugging together. I remember one particular occasion when at the end we were just so tired that we started doing round-robin when one of us was debugging and the other two were resting.

Generally, having a team saves time, you can learn from each other, share materials and grind some bonus points by creating bigger ensemble models.

You don't need a team

I feel obliged to say that team isn't needed. I have a friend who did both of the courses alone and always had at least the same amount of points.

Create an account on Pizza

You should do this immedietely. People will often post their inquiries about problems similar to yours. You should also use it to ask questions often. It all saves time.

I also recommend changing notification settings. The default is an email summary every 4-hours. This clutters mail too much and it doesn't add any benefit.

Do the first week assignments during the first week

There are usually two weeks for each assignment. You should do assignments the week they are assigned, not the week they are due. Waiting for the deadline is just stupid.

I can guarantee that there will come a week when you are just too tired or too busy which will make you skip otherwise an easy assignment which will result in you loosing points.

Don't lose easy points!

During the semester

The hardest thing you can do is skip the first 40 points and try to get the last 80.

The difficulty of assignments increases during the semester. So try-hard at the beginning, rest later.

Easiest bonus points are at the start

If you want bonus points from competitions the easiest time to get them is during the first one or two months of the course. At the beginning a lot of people are just trying to pass baselines and then stop improving their models. You should defeat them rather easily.

Later in the game, tryhards remain but baselines-only folks fall behind.

Watch/attend practicals

It's easy to get into the habit of skipping practicals. And in a way that's alright. They are recorded. But most of the time you will still watch them later because solving assignments without tips Milan Straka gives puts you in unnecessary disadvantage.

My usual procedure for solving an assignment consisted of the following steps (in order)

Neural network     \implies ensemble

Whenever there is a competition and you are using neural networks you should ensemble. It will give you a cheap nudge in performance which will probably result in at least one free bonus point. Don't let free bonus points escape.

Nevertheless, if you are doing ensemble of more than 5\approx 5 models it's probably too much. It's always better to have one well-finuted model than an ensemble of garbage.

Become a Grand Pullrequester

You can get up to 10 comunnity points for correcting slides, TODO comments in assignments, etc. It takes at most five minutes to correct a mistake if you have set up a fork of the original repository. On the other hand, solving an assignment can take several hours. Let's say that you need one hour for assignment-point and five minutes for slides-correcting-point. Thus, it is 12×12\times faster to get the slides-correcting-points.

If you've never contributed to public repository on GitHub here is how.

Read papers

Look for resources outside the course. A good start is to use references given during lectures. If you are looking for a reading on some specific topic you can also ask at Piazza. I am sure that somebody will be happy to help you.

Youtube might be also a good try. There are tons of high-quality courses from world-renowned universities that you can use to boost your studies.

The primary reason to study outside resources is that you'll learn much more, howbeit you can also draw inspiration from them. Mr. Straka allows you to use an existing architecture in a competition if you cite it properly.

Extend deadlines

Whenever you are trying to solve an assignment but can't do it in time or you are lost, write to Milan Straka.

Each time I asked for it, he was more than willing to help or extend the assignment deadline.

Exam

Before the exam, you have ideally at least 40 points and don't need to study super hard for it. Because the questions are known in advance you can prepare really well.

I gave the ML for Greenhorns exam 33 days and ended up losing 1111 points. 1010 of them were for a question I decided not to study and the last one was for forgetting to divide in some averaging. Nonetheless, easy 90+90+ points if we add points from practicals.

Generally, I would set aside 33 days for this exam. This should be enough even if you don't have that many surplus points. With a lot of points, you can also study less. I placed the Deep Learning exam at the end of the exam period and felt really burned out while studying for it. In the end I gave it a day and a half but because I had a lot of additional points (above 5050) it was enough.

More than 33 days might be overkill. One of my teammates studied for Deep Learning a little longer and he ended up having a perfect score :D.

Also don't feel bad if you don't have time to study every lecture. People skip them regularly. As I said I skipped a whole lecture during the ML for Greenhorns and four lectures for Deep Learning. However, during the Deep Learning exam it nearly backfired. I had a very unlucky hand in picking my test because 4040 percent of the exam was just about the last few lectures.

Doing it all

In the Deep Learning and the Deep Reinforcement Learning courses it is possible to pass by completing all assignments (as of writing this).

Mr. Straka says it's a little trap. It takes more time to do all the assignments than do some and then study for the test. In my opinion, studying for the exam is a nice refresher of all that was done during the semester so you might even learn more if you take it. Nonetheless, doing all assignments is cool, definitely possible, and in my Deep Learning course around a quarter of all students did so.

Have fun!

Don't think this point needs an explenation.

Good luck!