Sequence to sequence learning with neural networks
In his book "Thinking, Fast and Slow", Daniel Kahneman divides consciousness into two modes of operation: the reflective mode, where we act deliberately, learn, analyze new information, and the autopilot mode, where we rely on accumulated knowledge. The reflective mode requires significant effort, and we can stay in it only briefly before quickly becoming fatigued. In autopilot mode, we operate the rest of the time—it is effortless for us, but in this mode, we can make mistakes if we fail to recognize in time that the situation demands deliberate decision-making.
Large language models exhibit a similar property: they also operate in two modes—training mode and automatic response mode. In training mode, we train the model on a dataset or fine-tune it on a custom dataset. In question-answer mode, the neural network no longer adjusts its parameters; it uses accumulated experience to generate the expected response, just like a person in autopilot mode.
What if, in the future, neural networks learn to independently switch to training mode and train themselves? This would give rise to fully-fledged intelligences capable of adapting to new conditions and evolving. To achieve this, a large language model would need to be equipped with "senses" to process the surrounding reality and be allowed to expand and modify its own weight parameters. This is precisely what Ilya Sutskever speculated about in his talk. And this is exactly what we are now observing as we shift our focus from LLMs to AI agents. It seems this is currently the main driving force and the primary direction in the development of artificial intelligence.
A one Hundred Stars
Today, when I logged into GitHub, I noticed that someone had given my project "Simple Text Editor" another star. Although the project is simple and not very well-written, people seem to like it for some reason. I also like this project, and that's a good sign. The project's bug tracker is full of tickets about what can be improved, which means I've decided on the direction of work for the near future: I will be improving and developing it. If anyone else wants to help me with this, I would be very happy!
Invitation for participation in a study
Please help me test my hypothesis. I have a hunch that note-taking and note-taking correlates with people's levels of depression. My guess is that those who have a habit of writing everything down may, on average, be less unhappy.
Robert Sapolsky has a video on the topic of depression - in it, he talks about how antidepressants work. The most studied drugs regulate the mechanisms by which the brain regulates processes associated with the release and absorption of neurotransmitters. Very roughly, neurotransmitters are our “thought fuel” that we use to think. If you think a lot, the “thought fuel” runs out and the so-called ego depletion. We're starting to get dumb. To restore mental fuel, we need rest.
Large volumes of information cause increased consumption of “thought fuel,” causing exhaustion more quickly. Taking notes can reduce the load on our brain by shifting part of the function of memorizing and structuring information to external media. This means the brain uses fewer neurotransmitters, allowing it to do more. This makes a person more productive. This means that, in theory, this should also affect our level of depression or happiness. And if so, then it can be checked. For example, if you add a few additional questions related to note-taking to the Oxford happiness questionnaire, you can try to find a correlation in the answers. If there is a correlation, then people who take notes will, on average, be happier. Or they won’t, in case of reverse correlation.
If there is no correlation, then there is no trial. Either way, if the result turns out to be statistically significant, it would be a strong argument for why note-taking can help people who work in IT not only become more productive, but also happier. It may turn out that productivity is not about doing more, but about being happier.
Please help confirm or refute this hypothesis. There are only 35 questions in this questionnaire, and it should not take you much time. Thank you in advance!




When I was advised a book of Daniel Kahneman "Thinking, Fast and Slow", part of me arose: "How dare you point me out how should I think, fast, slow! My thinking area is only mine and nobody can point me out how I should do it!". As result the book was left unread.