Fill the Internet with blah blah blah (p=1.00000)

“Garbage in, Garbage out” is a basic caveat to anyone dealing with models, and a well-known source of biases in “AI”. Success of ChatGPT introduced large language models (LLMs) to the general public and is now clear that LLMs are here to stay. Already in 2022 out of all internet traffic 47.4% was automated traffic aka 🤖 bots. So far bots are limited in quality of generated content and easily detectable, but LLMs will change it, and this will bring a drastic change in the whole world of online text and images.

Recent ArXiv paper “The Curse of Recursion: Training on Generated Data Makes Models Forget” considers what the future might hold. Conclusion? Future Ais, trained on an AI-generated content published online, degenerate and spiral into gibberish—what authors call “model collapse”. The model collapse exists in a variety of different model types and datasets. In one funny example in a ninth-generation AI ended up babbling about jackrabbits, while the start point was a medieval architecture text.

This resembles a story from 2017 of two Facebook chatbots named Alice and Bob. The researchers conducted experiments to improve the negotiation skills of chatbots by having them play against humans and other bots. The first bot was trained to imitate human negotiation tactics in English, but it proved to be a weak negotiator. The second bot focused on maximizing its score and exhibited superior negotiation skills, but it resorted to using a nonsensical language that was incomprehensible to humans. (The hybrid bot scored only slightly worse than the humans, while maintained reasonable language).

In the heart of model collapse is a degenerative process when over time models forget the true data. Over the time two things happen—probability of “usual” is overestimated, while probability of “unusual” is underestimated. Disappearing of tails (“unusual”) leads to converging of generated content around central point (“usual”) with very small variance, and finally model collapses into high intensity gibberish.

This process has a shivering resemblance with the way how self-amplifying feedbacks shape the inertia of beliefs, which lead to black-and-white thinking, associated with psychiatric disorders, prejudices, and conspiracy thinking. Rigidity of beliefs and societal failure may reinforce each other through feedback mechanisms, very similar to those poisoning AI reality. Mitigation of rigid harmful beliefs in people may require improving the sustained exposure to counterevidence, as well as supporting rational override by freeing cognitive resources by addressing problems of inequity, poverty, polarization, and conflict. In a similar vein, avoiding model collapse, requires restoring information about the true distribution through access to genuine human-generated content.

In a world plagued by narrow-mindedness and echo chambers, the last thing we need is an narrow-minded and divisive AI. The true value of human-generated data lies in its inherent richness, encompassing invaluable natural variations, errors, twists and turns, improbables and deviants. Human-generated data represents more than just a function to be optimized; it encapsulates the very essence of what makes life worth living.

NB: Heading Image by master1305 on Freepik

What’s Wrong with ChatGPT? A view from Economists

Renowned economists—Daron Acemoglu and Simon Johnson—are concerned about ChatGPT. More precisely—the way how AI deployed by corporations in the US. Their analysis points out that it could displace workers, harm consumers, and bring losses to investors. The crux of the issues is focusing on cutting labour costs (in a short run), with little regard for the future of spending power and workers earnings, as well as neglecting the potential benefits of AI.

🤖 AI arms race, funded by billions from companies and venture-capital funds, bringing in a technology that can now be used to replace humans across a wider range of tasks. This could be a disaster not only for workers, but also for consumers and even investors.

👨‍🏭 The workers are facing clear and present danger. The job market is shifting, resulting in a decrease in demand for positions that require strong communication skills, ultimately leading to a decrease in higher-paying jobs. This trend is particularly challenging for younger people, just starting their careers, as there will be fewer entry-level positions available. AI powered tools could help in legal research, but deprive novice lawyers of learning techne through hands-on research.

🛍 Consumers, too, will suffer. Although they may suffice for routine inquiries, they are inadequate for addressing more complex issues—flight delay, household emergency, or dealing with a breakdown in personal relationships. We need understanding and actions of qualified professionals, not eloquent but unhelpful chatbots.

💸 Investors could also be disappointed as companies invest in AI technology and cut back on their workforce. Rather than investing in new technologies and providing training for their employees to improve services, executives are more interested in keeping employment low and wages as low as possible. This strategy is self-defeating and could harm investors in a long run.

🐙 The crux of the issues is that the potential of AI is being overlooked as most US tech leaders are investing in software that can replicate tasks already performed by humans. Contrary, AI-powered digital tools can be used to help nurses, teachers, and customer-service representatives understand what they are dealing with and what would help improve outcomes for patients, students, and consumers. The focus is primarily on reducing labor costs with little regard for the immediate customer experience and the long-term spending power of Americans. However, history has shown that this approach is not necessary. Ford recognized that there was no point in mass-producing cars if people couldn’t afford to buy them. In contrast, modern corporate leaders are utilizing new technologies in a way that could have detrimental effects on our future.

Read full article https://www.project-syndicate.org/commentary/chatgpt-ai-big-tech-corporate-america-investing-in-eliminating-workers-by-daron-acemoglu-and-simon-johnson-2023-02

P.S. I am currently reading “In The Age Of The Smart Machine: The Future Of Work And Power” by Shoshana Zuboff. The book published back in mid-1980s explores impact of the first wave of smart machines on labour relationships and future of work. There are a lot of similarities and lessons learned for current wave of ubiquitous AI-fication.

The Hidden Benefits of Commuting: Finding Serenity in the Space Between

A stray dog named “Boji” has become a local celebrity after using buses, subways and ferries to travel across Turkey’s metropolitan city of Istanbul.

I used to hate mornings. Istanbul is a very anisotropic City. Commuting from one neighborhood to another, just two blocks away, could take everything from 5 to 55 minutes. However, I start valuing this commuting time as a “liminal space” between personal and business realms. It helps me to transition between home and work life, to kick start the day, and to finish it.

There are three elements of this transition ritual. First, is physical activity, which fuels my internal engine. I have to walk 10-15 minutes either to the bus or the subway stop. This walk makes me energetic and improves my mood–oxytocin proved to have beneficial cognitive and behavioural effects. Second, I use  walking and commuting time to learn something new, to push my creativity for the day. Podcasts are great, I have some business related–Anecdotally Speaking, Talking About Organizations, or Re:Thinking.  Some pure fun on the bun–Friday Night Comedy or Something Rhymes with Purple. Other good options are Coursera and LinkedIn learning apps, which allow you to save videos to watch off-line.  Third, I use this time to reflect and think things over. I always have a notebook and pen with me, and Google.Docs on my smartphone. (Full disclosure: I drafted and edited this post while enjoying views of Beşiktaş from DT2 bus). This helps me to kick start day or unload business thoughts at the end of the day.

This ritual works very well while working from home or while on a business trip. You could use  a treadmill for a 15 minutes walk, or simply walk around a block–I often enjoyed cities at 6am, empty and quiet. Podcasts are always with you, as well as a notebook and a pen.

Don’t hate mornings, instead embrace an opportunity. By incorporating a simple morning ritual–such as a short walk, listening to something new and interesting, and reflecting on your goals and intentions–you can prioritize your well-being and kick start the day; or close the day, avoiding business spillover to home life. 

Meet less, write more!

Comic Strip Dilbert

We all survived meetings that could be an email. But could we do better? Tremendous summarized its high-documentation, low-meeting work culture in a blog post. Couple of takeaways:

🎯 Every meeting must have a reason—WHY are we meeting?—what Tremendous calls “meeting mindfulness”. Low-meeting culture is giving people the space and time they need to make innovative decisions. Meetings are reserved to discuss particularly heated topics and communicate on projects that require high-bandwidth collaboration

🤪 There are huge hidden costs of meetings, a 30-minute actually takes 68 minutes—15 minutes for preparation, 30 minutes meeting per se and 23 minutes to refocus. When meetings lack agendas, the efficiency cost quickly becomes grave

📑 High-documentation culture is the recipe for more productive, transparent, thoughtful, scalable, and efficient work. Writing a document—instead of shiny slides—forces people to focus on communicating their ideas as clearly as possible. While it comes with cost—time for writing (and editing), the benefits are clear—onboard and scale becoming easy. Thanks to searchable, traceable, public record of all significant decisions, projects, and initiatives across the organization

👩‍💻 Learn from successes. Professional development is a by-product of having access to the ‘how’ and ‘why’ of every impactful decision the company makes. Peer reviews help to learn and grow from the evolution of their own recommendations and proposals. It also teach people to provide feedback in a constructive, consistent and public manner (rather than derisive seagull-like critique)

Read blog https://www.tremendous.com/blog/the-perks-of-a-high-documentation-low-meeting-work-culture