Premise
I’ll run a small experiment: I’ll start summarising things that interested me during the week, and see for how many weeks I can continue before dropping this moment in favor of one of the many other priorities. In other words, let’s see if this task is useful (and pleasant) for me!
Playing with LLMs
I’ve been experimenting with large LLMs, Llama 7B in particular. I wrote some scripts and ran them locally on my laptop. Pytorch doesn’t run well on M1: the GPU is supported, but many frameworks require CUDA. I attempted various strategies to do fine-tuning, but it was impossible. Even smaller models require too much memory. Having to hack Pytorch didn’t help. I also had no luck with (free) cloud services. Google Colab is great, but it’s not designed for that.
On the positive side, I had lots of fun with LlamaIndex. I quickly wrote an application that indexes raw documents and answers questions using them. I used Llama 7B and it’s extremely slow, but there is space for improvement. This paper in particular captured my attention.
More on LLMs
Studies use AI to improve software. That’s certainly interesting, but, in my everyday job, I find so many areas of improvements just by reviewing the code that I wonder how much we really need the help of AI and how much we simply lack basic software engineering knowledge.
AI in general
It’s true that AI and simple automation can solve any sort of problem. However, adopting these technologies comes at a cost. Although it’s possible to have automated valet parking robots, it is reasonable to say that they won’t be massively adopted until their cost and efficiency make them more convenient than a low-skilled person.
In contrast, news of the week is Axel Springer replacing journalists with AI. It will be interesting to observe what’s going to happen. AI can dramatically speed up research and review articles (LLMs can do fact-checking!) so we may improve press quality or it may become cheaper. I guess that AI will kill poor-quality newspapers but high-quality ones won’t disappear, just like mass production didn’t kill top-quality craftsmanship.