Working with data processing
pipelines made me see some analogies with the process of human learning. When a human being learns a new skill, they usually go through the state which could be called “snapshot reload” in the data processing pipelines lingo. That means, both a human and data pipeline have to spend some time accruing the “initial set of knowledge” based on which they will (hopefully) become useful in the future. Once this happens (e.g. you as a human finished a school or a professional course), in order to be fully useful one would also need to keep their knowledge current (get the recent information related to that set of skills by following the related events, reading articles, trying out the new tricks, etc etc). In pipeline world, this is called “catching up on the incrementals”. If this is not being done (or takes too much time) the pipeline (or a human) might need to do the new “snapshot reload” (go back to school) to stay in business.
What’s important to understand is that every such system is essentially useless and vulnerable while it’s doing those costly reloads. Almost everyone can relate to their student times as the periods of high uncertainty and remember the efforts it sometimes costed to stay on track, with no finish line in sight. The price of falling at that stage is also very high: dropping out is easier than restarting the process, and can result in hard compromises regarding the final state. Software, designed in a waterfall way and discontinued before completion, is useless (except for whatever could be scavenged for reuse in the next projects). A pipeline which failed to reload its state gets restarted from scratch, with no advantage from the incomplete work. And though a few school dropouts might become billionaires, the rest would eventually have to get their life back together and either return to that or other school or significantly lower their future expectations.
The traditional education was (and is) all about spending long (and costly) time in the “snapshot reload” phase (school, university, then perhaps more time in academia as a postgraduate) and because it’s recognized as being very time-consuming and costly, changing jobs or learning new skills as an adult is considered to be unlikely. (You can also draw an analogy with
waterfall process here).
Ideally, though, both in the data processing world, in the software development in general and in the human world, one would prefer for things to become useful immediately, without going through that painful and costly (and often black-box) phase mentioned above. There are numerous
papers describing large-scale incremental data flows. There is
Agile methodology, which is about developing software in an incremental way, as opposed to phased Waterfall style. And nowadays, there is a pronounced tendency to try learning new skills incrementally: most of the
online learning courses and the variety of learning apps are about splitting the information to be learned into piecemeal bites and feeding it to the students on a regular basis, assuming they would gradually become proficient in the given skill.
The big question is, is it always going to work or would the process in some cases be too slow to be useful? Could every child completely skip the hours in school and become the high-functioning adult just by incremental learning, interleaved with their daily life? Could every adult gradually shift to a new role, and if so, at which point of that shift can they actually become useful? Or are the “snapshot reload”, high-focus-on-the-problem-at-hand phases inevitable?
To me, these questions largely seem unanswered, and the second one might be a weak point of the online learning in general. It seems currently be well optimized for monetization via getting students to pay, but less so for monetization via helping the learners to refocus by steering them into using their acquired knowledge and connecting with the relevant communities. Udacity is making steps in that direction by trying to place their students into
relevant job roles, but this looks more like an exception than a rule so far.
I wonder whether there isn’t also some unconscious bias from the part of those who did go through the traditional “snapshot reload” phase regarding those who obviously didn’t, as incrementally acquired knowledge might appear “less valid” because it misses the hard proof of completing the “snapshot reload” phase and hence the credibility - which might be not ungrounded, on one side; on another side, the existence of scientific titles with the suffix “honoris causa”, “for the merit” proves that it was always posssible to do things differently.
The
interview with Freeman Dyson, which mentions not obtaining the PhD due to not believing in the system, was my most recent reminder on that. In general, my hope is that the future becomes more fluid and "incremental" and will allow people to switch between different activities as they go through their lives, without discarding their ideas just because they arrived "too late" to be carried through a necessary snapshot reload phase before getting applied somewhere.
