Sequence to sequence learning with neural networks
In his book "Thinking, Fast and Slow", Daniel Kahneman divides consciousness into two modes of operation: the reflective mode, where we act deliberately, learn, analyze new information, and the autopilot mode, where we rely on accumulated knowledge. The reflective mode requires significant effort, and we can stay in it only briefly before quickly becoming fatigued. In autopilot mode, we operate the rest of the time—it is effortless for us, but in this mode, we can make mistakes if we fail to recognize in time that the situation demands deliberate decision-making.
Large language models exhibit a similar property: they also operate in two modes—training mode and automatic response mode. In training mode, we train the model on a dataset or fine-tune it on a custom dataset. In question-answer mode, the neural network no longer adjusts its parameters; it uses accumulated experience to generate the expected response, just like a person in autopilot mode.
What if, in the future, neural networks learn to independently switch to training mode and train themselves? This would give rise to fully-fledged intelligences capable of adapting to new conditions and evolving. To achieve this, a large language model would need to be equipped with "senses" to process the surrounding reality and be allowed to expand and modify its own weight parameters. This is precisely what Ilya Sutskever speculated about in his talk. And this is exactly what we are now observing as we shift our focus from LLMs to AI agents. It seems this is currently the main driving force and the primary direction in the development of artificial intelligence.
A one Hundred Stars
Today, when I logged into GitHub, I noticed that someone had given my project "Simple Text Editor" another star. Although the project is simple and not very well-written, people seem to like it for some reason. I also like this project, and that's a good sign. The project's bug tracker is full of tickets about what can be improved, which means I've decided on the direction of work for the near future: I will be improving and developing it. If anyone else wants to help me with this, I would be very happy!
Invitation for participation in a study
Please help me test my hypothesis. I have a hunch that note-taking and note-taking correlates with people's levels of depression. My guess is that those who have a habit of writing everything down may, on average, be less unhappy.
Robert Sapolsky has a video on the topic of depression - in it, he talks about how antidepressants work. The most studied drugs regulate the mechanisms by which the brain regulates processes associated with the release and absorption of neurotransmitters. Very roughly, neurotransmitters are our “thought fuel” that we use to think. If you think a lot, the “thought fuel” runs out and the so-called ego depletion. We're starting to get dumb. To restore mental fuel, we need rest.
Large volumes of information cause increased consumption of “thought fuel,” causing exhaustion more quickly. Taking notes can reduce the load on our brain by shifting part of the function of memorizing and structuring information to external media. This means the brain uses fewer neurotransmitters, allowing it to do more. This makes a person more productive. This means that, in theory, this should also affect our level of depression or happiness. And if so, then it can be checked. For example, if you add a few additional questions related to note-taking to the Oxford happiness questionnaire, you can try to find a correlation in the answers. If there is a correlation, then people who take notes will, on average, be happier. Or they won’t, in case of reverse correlation.
If there is no correlation, then there is no trial. Either way, if the result turns out to be statistically significant, it would be a strong argument for why note-taking can help people who work in IT not only become more productive, but also happier. It may turn out that productivity is not about doing more, but about being happier.
Please help confirm or refute this hypothesis. There are only 35 questions in this questionnaire, and it should not take you much time. Thank you in advance!
Smart Plug from Ikea
In my recent shopping tour to Ikea I bought a couple of new devices to extend my smart home zoo: smart socket and a couple of smart bulbs TRADFRI. Smart socket I connected to towel dryer in the bathroom and smart bulbs put to the light spots under the bath mirror.
Before the towel dryer was connected through some no name chinese smart plug which could be controlled through WiFi. From time to time there were problems with this smart plug since it was an online device that was dependent on some unknown Chinese servers located somewhere. When there was no internet in my apartment or these servers were unavailable the smart plug could not switch off or switch on. If the towel dryer was off for some time it was causing just stinky musty towels. If the towel dryer was on for some time it would turn my small bathroom into a small nice sauna. Also I had no idea how to integrate this device to my smart home setup so I decided to replace it with the more predictable device. My choice has fallen to the Ikea model E1702 which can work as a zigbee router which can amplify a signal of my DIY zigbee2mqtt hub and has a price just 10 euro which is much cheaper than most competitors.
Smart Plug was automatically recognized and automatically integrated to my zigbee2mqtt system but light bulbs could not join by some reason. The model LED2104R3 is too new and there is still no configuration for this device available yet. I will investigate how I can fix it later.
The first task that I had to solve was how to switch on and off my towel dryer at specific times as it was done before with an old smart plug. The zigbee2mqtt can control devices in real time but it is all that it can do. There is no event scheduler to switch devices on and off at specific times. The simplest way is just send control events to the MQTT bus from cron, but I decided that it would be too low level of a solution. There are plenty of Open Source No code solutions that can be suitable for this job. My favorite is Node Red since it is very lightweight and can run on any device event on Parpberry Pi Sero W! Since I already had the Node Red application installed to Raspberry Pi Zero W, I decided to use it to do the job.
Fortunately this is very easy and does not require many steps. First, I had to install an extension node-red-contrib-zigbee2mqtt-devices. This can be done through the web interface. Once the extension was installed all that is left just to put some elements to the workflow and wire them together.
There is the final schema that works for me:
Now my towel dryer is turning on for some time twice a day in the morning and evening which is just enough to keep towels dry and does not cause high electric bills. Additionally it can be contolled using a web page. It doesn't require an internet connection, all it needs just an internal WiFi because Node Red and zigbee2mqtt are installed on two different Raspberry Pi Zero W devices.
Let's see how well it will work in the long run.