2024-12-18

Sequence to sequence learning with neural networks

In his book "Thinking, Fast and Slow", Daniel Kahneman divides consciousness into two modes of operation: the reflective mode, where we act deliberately, learn, analyze new information, and the autopilot mode, where we rely on accumulated knowledge. The reflective mode requires significant effort, and we can stay in it only briefly before quickly becoming fatigued. In autopilot mode, we operate the rest of the time—it is effortless for us, but in this mode, we can make mistakes if we fail to recognize in time that the situation demands deliberate decision-making.

Large language models exhibit a similar property: they also operate in two modes—training mode and automatic response mode. In training mode, we train the model on a dataset or fine-tune it on a custom dataset. In question-answer mode, the neural network no longer adjusts its parameters; it uses accumulated experience to generate the expected response, just like a person in autopilot mode.

What if, in the future, neural networks learn to independently switch to training mode and train themselves? This would give rise to fully-fledged intelligences capable of adapting to new conditions and evolving. To achieve this, a large language model would need to be equipped with "senses" to process the surrounding reality and be allowed to expand and modify its own weight parameters. This is precisely what Ilya Sutskever speculated about in his talk. And this is exactly what we are now observing as we shift our focus from LLMs to AI agents. It seems this is currently the main driving force and the primary direction in the development of artificial intelligence.

 

Оставить комментарий

 
  2024-07-06

A one Hundred Stars

Today, when I logged into GitHub, I noticed that someone had given my project "Simple Text Editor" another star. Although the project is simple and not very well-written, people seem to like it for some reason. I also like this project, and that's a good sign. The project's bug tracker is full of tickets about what can be improved, which means I've decided on the direction of work for the near future: I will be improving and developing it. If anyone else wants to help me with this, I would be very happy!

 

Оставить комментарий

 
  2024-03-01

Invitation for participation in a study

projects

Please help me test my hypothesis. I have a hunch that note-taking and note-taking correlates with people's levels of depression. My guess is that those who have a habit of writing everything down may, on average, be less unhappy.

Robert Sapolsky has a video on the topic of depression - in it, he talks about how antidepressants work. The most studied drugs regulate the mechanisms by which the brain regulates processes associated with the release and absorption of neurotransmitters. Very roughly, neurotransmitters are our “thought fuel” that we use to think. If you think a lot, the “thought fuel” runs out and the so-called ego depletion. We're starting to get dumb. To restore mental fuel, we need rest.

Large volumes of information cause increased consumption of “thought fuel,” causing exhaustion more quickly. Taking notes can reduce the load on our brain by shifting part of the function of memorizing and structuring information to external media. This means the brain uses fewer neurotransmitters, allowing it to do more. This makes a person more productive. This means that, in theory, this should also affect our level of depression or happiness. And if so, then it can be checked. For example, if you add a few additional questions related to note-taking to the Oxford happiness questionnaire, you can try to find a correlation in the answers. If there is a correlation, then people who take notes will, on average, be happier. Or they won’t, in case of reverse correlation.

If there is no correlation, then there is no trial. Either way, if the result turns out to be statistically significant, it would be a strong argument for why note-taking can help people who work in IT not only become more productive, but also happier. It may turn out that productivity is not about doing more, but about being happier.

Please help confirm or refute this hypothesis. There are only 35 questions in this questionnaire, and it should not take you much time. Thank you in advance!

Click here to start a survey

 

Оставить комментарий

 
  2023-12-21

Smart Plug from Ikea

In my recent shopping tour to Ikea I bought a couple of new devices to extend my smart home zoo: smart socket and a couple of smart bulbs TRADFRI. Smart socket I connected to towel dryer in the bathroom and smart bulbs put to the light spots under the bath mirror.

Before the towel dryer was connected through some no name chinese smart plug which could be controlled through WiFi. From time to time there were problems with this smart plug since it was an online device that was dependent on some unknown Chinese servers located somewhere. When there was no internet in my apartment or these servers were unavailable the smart plug could not switch off or switch on. If the towel dryer was off for some time it was causing just stinky musty towels. If the towel dryer was on for some time it would turn my small bathroom into a small nice sauna. Also I had no idea how to integrate this device to my smart home setup so I decided to replace it with the more predictable device. My choice has fallen to the Ikea model E1702 which can work as a zigbee router which can amplify a signal of my DIY zigbee2mqtt hub and has a price just 10 euro which is much cheaper than most competitors.

Smart Plug was automatically recognized and automatically integrated to my zigbee2mqtt system but light bulbs could not join by some reason. The model LED2104R3 is too new and there is still no configuration for this device available yet. I will investigate how I can fix it later.

The first task that I had to solve was how to switch on and off my towel dryer at specific times as it was done before with an old smart plug. The zigbee2mqtt can control devices in real time but it is all that it can do. There is no event scheduler to switch devices on and off at specific times. The simplest way is just send control events to the MQTT bus from cron, but I decided that it would be too low level of a solution. There are plenty of Open Source No code solutions that can be suitable for this job. My favorite is Node Red since it is very lightweight and can run on any device event on Parpberry Pi Sero W! Since I already had the Node Red application installed to Raspberry Pi Zero W, I decided to use it to do the job.

Fortunately this is very easy and does not require many steps. First, I had to install an extension node-red-contrib-zigbee2mqtt-devices. This can be done through the web interface. Once the extension was installed all that is left just to put some elements to the workflow and wire them together.

There is the final schema that works for me:

Now my towel dryer is turning on for some time twice a day in the morning and evening which is just enough to keep towels dry and does not cause high electric bills. Additionally it can be contolled using a web page. It doesn't require an internet connection, all it needs just an internal WiFi because Node Red and zigbee2mqtt are installed on two different Raspberry Pi Zero W devices.

Let's see how well it will work in the long run.

 

Оставить комментарий

 
  2023-10-19

BaselOne

I visited a conference called BaselOne. The conference was amazing! Together with Karsten Thoms, we presented a talk about modernizing legacy infrastructure. It was a very exciting experience for me.

I met many very interesting people from GitHub, JetBrains, Gradle, jOOQ, and many others.

The conference world is very interesting and motivating. It feels like the world of conferences is very special. Many speakers were really enjoying speaking there! I gained great motivation to move forward, discovered directions to grow, and got many insights about public speaking.

 

Оставить комментарий

 
  2023-06-22

GPT-4

I have installed GPT4 locally today. Now I understand the reason for the hype that has swept the world today.

It seems we accidentally reinvented a human.

See yourself, according to different estimations, the amount of memory of the human brain ranges from 10^6 to 10^16 bits. This, translated into a programming language, ranges from 122KB to 1136TB. Let's assume that the truth lies somewhere in the middle and is 10^11 bits, which is, 11 Gb.

The size of the model that I downloaded from the Internet in order to run the chat shown in the screenshot weighed about 4 Gb. And this is just one of the many free models available in the app. Some models are much larger. This is the same order as the rough average estimate of the memory capacity of the human brain!

We have created a repository of human experience for the transfer of which from person to person does not need an intermediary in the form of another person! We have learned how to save experience not in the form of books or oral retelling, but in a completely new form - in the form of a knowledge model that can be run on a computer and instantly get an answer to the question posed, and that's cool! We got a brain in a jar that doesn't have the disadvantages of an organic brain that is always connected, always in a good mood and ready to help.

Of course, not everything is so rosy yet, a number of questions arise, for example, how to verify the answers received, how to effectively use such models, I'm not talking about all sorts of questions about how to deal with fighters for ethics, copyrights and other miscellaneous...

Pandora's box is open, it seems to be with us now for a long time ...

 

 

Оставить комментарий

 
  2023-03-30

The Secret 5 AM Club

"The 5 AM Club" is a self-help book written by Robin Sharma, a well-known leadership expert and author. The book focuses on the idea that waking up at 5 AM can lead to increased productivity, focus, and happiness.

The book tells the story of a struggling artist, a burnt-out entrepreneur, and a beggar vagabond who all attend a couching lection led by a mysterious and wise mentor named The Spellbinder. Very soon appears that beggar vagabond is not the one who he pretents to be. This event drasticaly changes live of these characters.

The book is well-written and engaging, and the author provides practical advice and exercises for readers to implement in their own lives. However, some readers may find the book's emphasis on waking up at 5 AM to be unrealistic or difficult to achieve.

Overall, "The 5 AM Club" is a helpful and inspiring read for those looking to improve their productivity and overall well-being, but readers should approach the book with an open mind and willingness to make significant changes to their daily routines.

I haven't come across a more inspiring book in a long time. Robin Sharma knowingly has the fame of a person who knows how to motivate. After Jonathan Livingston Seagull, I thought I couldn't find a more positive book. It's nice to know that I was wrong.

P. S. ChatGPT helped me to to write this blog post, — awesome tool that will change out lives very soon.

 

Оставить комментарий

 
  2023-01-11

Thinking, Fast and Slow

When I was advised a book of Daniel Kahneman "Thinking, Fast and Slow", part of me arose: "How dare you point me out how should I think, fast, slow! My thinking area is only mine and nobody can point me out how I should do it!". As result the book was left unread.

Later however, when I have seen the video of  Tim Urban on Ted Talk, which I knew later was inspired also by this book, I finally decided to read this book.

It looks like Kahneman made his book hard to read it for some purpose. Looks like he really wanted to make his book possible unknown and hard to be understood. Instead of prominent characters of Tim Urban with characters like Monkey, Monster and Rational Type in his book Kahnemann uses terms "System 1" and "System 2". Those terms are not easy to remember.

There is a idea that the best texts are written the by most awkward language.

The Kahneman himself says that he uses this way to make people pay more attention to details because according his research when information is harder to be understood this activates the brain of the reader and makes people more attentive and critical. Probably it is the case, but also it opens opportunities for other authors use Kahneman's book to create more viral derivatives.

This book does not teach you how to think. It does not make speed of your thinking faster or slower. This book brings you other ideas about the way how people are using their brain. Nowadays his work is being used by marketologists, traiders, economists and all the people who are dealing with behaviour of higher primates e.g. humans.

In the 1970s, two assumptions were generally accepted. First, that people are basically rational and sane. Secondly, that most deviations from rationality are explained by emotions. But Kahneman's research has shown that people's constant thinking errors are due more to the very mechanism of thinking than to the influence of emotions. Today, researchers agree with the thesis that our minds are prone to systematic errors. Kahneman's main idea is to demonstrate the workings of the mind, taking into account the latest discoveries in cognitive and social psychology.

Should you read this book? Decide for yourself. But if we decide to read it be ready that your world will never be the same!

 

Оставить комментарий

 
  2022-12-12

The Third Smart Contract challenge

The people from TON Foundation organised again a contest. This time we need to write 5 smart contracts on Func language.

As they say the challenge is purposed to encourage the developers learning Func language and should be relative simple. For me thought is not such simple. I need to refresh all my knowledge about math and computer systems in general. I can not even understand WHAT should be done, not mentioning of HOW it could be implemented.

For example is the task 4 one need to implement an operation to sum of two elliptical curves Curve25519.

I assume that among readers of my blog there are someone who is calculating elliptical curves for fun every day before breakfast, but I am not like that.

However I starting to feel that I like this process - to dig into the new area and revealing the new ideas and knowledge. Nikolay Durov made a great deal of creating such an infrastructure. Someone even call him a second Nakomoto.

So far I have more questions that answers. For example: The TON smart contracts are mutable. How we can prevent scam then if the creator of the contract can rewrite it later? Or, there is an another example: wallets in TON network are smart contracts. There are some of them are written on Func, some on Fift. In the documentation they say that these wallets are almost the same. Why then the Func code of smart contract is not available?

In order to understand how it works one need to read trough hundreds of pages of documentation. There as some step-by-step guides but they are really different, opinionated and very often outdated.

If anybody love to solve puzzles, I would really recommend to participate in the challenge. The first phase is unfortunately already over, the second phase will be running till 21th December, so there are still some time left.

 

Оставить комментарий

 
  2022-09-11

How to learn to work with Vim?

Perhaps there is no other text editor you need to work with. study specifically. However, if you learn to work with Vim, then a person gets amazing advantages over other people. Vim is usually already installed on most Linux systems and basic skills working with it are simply necessary for most developers.

Because this editor has its origins in the times when there was no Internet at all, the mouse had not yet been invented, and computers were terminals with a cathode ray tube and a keyboard that did not have arrow keys, then working with this editor looks akin to the dances of a shaman for the uninitiated.

One of the important advantages of this editor is the plugin system. It is thanks to this system that it is possible to transform Vim like an IDE, get syntax highlighting, auto-suggestions, file system navigation and much more.

There are many resources to help you learn how to use this editor, but I thought, if vim has so many plugins, is there a plugin for learning functions the editor himself? And it turned out that yes, there is!

Is is called vim-tutor-mode! In order to install it you need first to install vim-plag - it is an extention manager for vim. Then in a file ~/.vimrc you should add this line: "Plug 'fmoralesc/vim-tutor-mode'" to finalize plugin installation call: :PlugInstall

If the plug in installed successfully, you can start it with command :Tutor.

The tutorial covers the basic functionality of the editor, including working with files, searching and replacing, editing, running operating system commands, and more.

The passage took me several days in small sessions of about an hour a day. Passed on the phone in Termux, on the computer it should be even faster.

 

Оставить комментарий