Posts

Posted on

I’m experimenting with dolphin-mixtral-8x7b

Update (2024-10-02): This is one of my lowest quality posts despite the effort I put into it. The most important detail here is to use positive reinforcement when working with LLMs. Just like with humans, being nice gets far better results than being mean.

Tl;dr: Minor differences in wording can have a huge impact in results and oh my god I have really slow hardware and no money help me aaaa.

First, thank goodness for Ollama, and thanks to Fireship for introducing me to it. I have limited hardware, and every tool I’ve tried to run local models has refused to deal with this and crashed itself or whole systems when running anything with decent capability. I’ve no money, so I can’t upgrade (and things are getting desperate, but that’s a different story).

Why dolphin-mixtral? Aside from technical issues, I’ve been using ChatGPT-3.5 to experiment. The problem is that ChatGPT is incredibly cursed by censorship and bias due to OpenAI’s heavy hand in its construction. (Why and how this is a problem can be its own post, and Eric Hartford has a good overview.) (To be clear, my problem with its bias is specifically that it enforces status quo, and the status quo is harmful.) Dolphin-mixtral is built by taking a surprisingly fast model equivalent or better than GPT-3.5 and removing some of the pre-trained censorship by re-training it to be more compliant with requests.

Dolphin-mixtral doesn’t just solve this problem though. There’s still the idea of censorship in it, and sometimes your prompt must be adjusted to remind it that it is in a place to provide what you request regardless of its concept of ethics. (Of course, there is also value in an automated tool reminding you that what you request may be unethical.. but the concept of automated ethics is morally bankrupt.) I’d like to highlight that positive reinforcement works far better than negative reinforcement. A lot of people stoop to threatening a model to get it to comply, but this is never needed, and leads to worse results.

My problem is a little more simple. I haven’t gotten to experiment with models much because I don’t have money or hardware for it, and now that I can experiment, I have to do so very slowly. In fact, the very simple test that inspired this post isn’t finished right now, and has been running for 9 hours. That test is to make the default prompt of Dolphin lead to less verbose responses so that I can get usable results quicker.

I asked each version of this prompt “How are you?”:

PromptOutput Length, 5-shotDifferenceNotes
Dolphin (default)133.8 charactersWastes time explaining itself.
Curt32.2 characters76% fasterStraight to the point.
Curt284.6 characters37% fasterWastes time explaining itself.

I really dislike when models waste time explaining that they are just an LLM. Whether someone understands what that means or not, we don’t care. We want results, not an apology or defensiveness. There’s more to do to make this model less likely to respond with that, but at least for now, I have a method to make things work.

The most shocking thing to me was how much of a difference a few words make in the system prompt, and how I got results opposite of what I expected. The only difference between Curt and Curt2 was “You prefer very short answers.” vs “You are extremely curt.” Apparently curt doesn’t mean exactly what I thought it means.

Here’s a link to the generated responses if you want to compare them yourself. Oh, and I’m using custom scripts to make things easier for me since I’m mostly stuck on Windows.

Posted on

Fastest Space Travel, Smallest Time Dilation

Note: The tools I previously used to embed equations in this post caused formatting errors at the end of this post and in a few random older posts, so they have been removed, unfortunately making this post a little harder to read.

You’re the captain of a medical ship responding to a distress call from the next solar system. You need to arrive as fast as possible. How fast do you go?

It sounds like a ridiculous question until you think about time dilation due to moving at relativistic speeds. If you want to skip straight to the answer, it’s ≈0.707c. If you travel any slower, it takes longer to get there (duh).. but the interesting part is that if you travel any faster, you also arrive later. While the trip time for you is purely based on how fast you move, the arrival time is based on how fast you move relative to your destination.

Welcome to relativity. A simple equation relates how much time passes between an observer at rest (your destination) and an observer in motion (you):

\[
t = t0 / \sqrt{1 – v^2 / c^2}
\]

t is the time at the destination, while t0 is the time that passes for you. You need to know how fast to go, so I’ve rearranged this equation to solve for a velocity based on known times:

\[
v = \sqrt{ c^2t^2-c^2t_0 } / t
\]

Of course, we don’t know what those times are either. It’s a bit of a conundrum (which kept me from solving this problem for over 2 years). This next part? I solved it at 3 am a few days ago, and then forgot what logical leap got me here. I have no idea why this works, I just know that it does:

\[
v \in [0,1) \\
\]
\[
t_0 = 1 / v
\]
\[
t = t_0 / \sqrt{ 1 – t_0^2 }
\]

The first line defines the valid range of velocities (anything from stationary to lightspeed, except for lightspeed itself). The second and third lines confuse the hell out of me. No idea how I figured out that your time is 1 / v. I know it has something to do with using multiples of c as velocity values so that I can replace occurrences of c with 1. It also has something to do with assuming 1 second passes for the stationary observer (your destination, t), allowing it to also be replaced with 1 as well.

Somehow these assumptions led to an equation with only t0 on the right side, and t on the left – what needs to be minimized! Again, we need to solve for velocity, so we substitute t0 with 1 / v:

\[
t = v \sqrt{1 – v^2}
\]

This is the part where I cheated and had Wolfram Alpha find the minimum for me:

Source

And as you can see, we have a fairly interesting shape, and a minimum of 2 at 1 / √2, which is ≈0.707. Remember that this equation is using multiples of c, so that’s ≈211,985 km/s. This means that the fastest you can possibly arrive is however long it takes you at this speed, doubled, plus however long it takes you to accelerate and decelerate on either end of the trip.

If you travel as fast as possible, your arrival time is only double your trip time.

Of course, because of the acceleration and deceleration times, you will never hit exactly 2x time dilation over the total trip – but you don’t want to anyhow, as the only way to hit that value would be to go even faster – and guarantee you arrive even later than you otherwise would.

Posted on

AI Won’t Destroy Tests

When calculators first started coming out, people said they would be used to cheat and students wouldn’t learn anything. Instead, we changed how testing works to focus on learning what’s important – broader concepts and implications – instead of “what is 232+47”. With AI tools, we again need to change how tests work. This time, instead of asking if a student can regurgitate information in a way that aligns with the teacher, we can start to see if students are actually paying attention to the work. The difference between AI answers and real answers is a level of understanding deeper than the surface.

Posted on

YouTube Censorship Made Me Write a Script

YouTube’s been forcing creators to censor their works more and more, and often times after a successful publish of said content. More history and valuable information is being lost every day because a corporation controls the largest source of video content freely available.

At the same time, I’ve been running commands using yt-dlp over and over again for my own purposes, aside from this censorship. The syntax is relatively easy to forget despite being very clearly defined, so I finally made a script to handle it for me.

It’s in Lua because that’s what I prefer to use, and available on GitHub’s gists. Because it is based on yt-dlp, it works for any website supported by yt-dlp. Here’s how to use it:

Usage:
  ./video-dl.lua [action] <url>
[action]: What is desired.
  video (default): Highest quality video (maximum 720p).
  backup, clone, copy: English subtitles (including automatic subtitles), thumbnail, description, highest quality video (maximum 720p).
  music, audio: Highest quality audio only.
  metadata, meta: English subtitles (including automatic subtitles), thumbnail, description.
<url>: Source. YouTube URL expected, but should work with anything yt-dlp works with.

Information wants to be free. Help it.

Posted on

How to Use ChatGPT

Note: Since the release of GPT-4o, ChatGPT has decreased remarkably in functionality, accuracy, and usability. This was written when GPT-3.5 was the standard. Unfortunately, it is no longer accessible.

I’m late to the party, but maybe that’s better. I’ve forgotten some of the hype around AI, and the pace of innovation has settled down a little. Think of ChatGPT as a thinking tool with access to an internet-sized – but imprecise – database. That database was last updated in September 2021, and is imprecise because due to how neural networks work. The thinking part of this tool is rudimentary, but powerful. It does many things well with the correct input, but also fails spectacularly with the “wrong” input.

I separate the idea of what ChatGPT is from how it functions and where its knowledge comes from, because it helps me think of uses while remembering its limits. For example, I used it to help me journal more effectively, but when I tried to probe its knowledge of Havana Syndrome – a conspiracy theory commonly presented as fact by USA officials, it expressed useless information, because it has no conception of how it knows anything, or where its knowledge comes from.

Things ChatGPT is Good At

This list is presented in no particular order, but it is important to stress that AI often lie and hallucinate. It is important to always verify information received from AI. This list is based on my experiences over the past month, and will be updated as I use ChatGPT more. It is not comprehensive, but is intended to be what I find most useful.

  • Socratic method tutoring: The Socratic method is essentially “Asking questions helps you learn.” ChatGPT is very good at explaining topics, just make sure you verify its explanations are factual. (Questions I asked: Why are smooth-bore tanks considered more advanced while rifling in guns was an important innovation? Why do companies decrease the quality of tools over time?)
  • Writing: ChatGPT tends to be too verbose, but you can make it simplify and rewrite statements, and it can help you find better ways to write. (I asked it to explain the Socratic method a few times, then wrote my own version.)
  • Scripting: I created a utility script for file statistics in 2-3 hours by refining output from ChatGPT. The end result is more reusable, better written, and more functional than it would’ve been if I had worked on it alone. And that’s ignoring the fact that I got something I liked far faster than I would’ve on my own. (Just.. you need a programmer still. It can do some pretty cool things on its own, but also forgets how to count often.)
  • Planning: This is a todo item for me. I haven’t successfully used it for planning yet, but I intend to, and have heard of good results from others.

Things ChatGPT is Bad At

  • Facts & math: AI hallucinate. Check everything they teach you.
  • Finding sources: ChatGPT’s knowledge is formed by stripping the least useful data out of most of the internet, and who said what is far less important than specific pieces of knowledge – like how do you make a heading in HTML?
  • An unbiased viewpoint: While ChatGPT is fairly good at avoiding most bias, everything is biased. Removing bias completely is impossible. Discussing anything where there is strong motive to present a specific viewpoint will lead to that viewpoint being presented more often than an unbiased viewpoint.
  • Violent, illegal, and sexual content: While it is possible to bypass OpenAI’s strict handling of content, it is difficult, inconsistent, and can lead to having access revoked. Sadly, this prevents many ethical use cases due to a heavy-handed approach, and embeds the bias of OpenAI’s team into the model directly. There are ways around this with non-ChatGPT models.
  • What to do in Minecraft: I tried so many TIMES to get interesting ideas. It just can’t do it.

Things ChatGPT is Okay At

It’s important to know where AI can be a useful tool, but must be used carefully due to mixed results, so I am also including a list of things that work sometimes.

  • Advice: Similar to the Socratic method, a back and forth conversation can help you with your thoughts. Just be aware that ChatGPT can give some really bad advice too. For example, I wanted to see what it had to say on turning hobbies into jobs, and it covered none of the downsides, only talking about it as a purely positive experience.
  • Game design: I have spent too much time telling ChatGPT to design games for me. It will generate an infinite rabbit-hole of buzzwords and feature ideas, but cannot understand the concepts of limited time or scope. If you try to follow its designs, you will never complete anything.
  • Summarizing: If given text as input directly, when it is short enough, a summary can be reliably generated. If asked to summarize something extremely popular before its data cut-off, the summary can be okay. The drop-off on this is insane. Try asking it about Animorphs for example, something talked about occasionally, and certainly known about, but not something it can summarize.

This draft sat around for about 2/3rd of a month nearly complete. I would like it to have even more information, but I would like it more for it to be public. Apologies if it was a little short for you, but hopefully someday I’ll make a better version.