When Open Source Maintainers Don’t Understand Community is Important

This is just to vent frustration at a thoroughly stupid experience I had recently. A portion of that stupidity is me failing to read something correctly, but I’m just really stuck on the stupidity of the response to me asking for help:

I asked for clarification, and was told to go away.

My reaction clearly indicates that I am not undrstanding something, and I even tried to give context to where I’m coming from so that it would be easier to spot what I misunderstood, but instead I was told to go ask a bot.

And then they blocked anyone from ever asking for help again.

The public is not allowed to open issues now.

What’s most frustrating to me about this is that it coincides perfectly with another issue I ran into today where I couldn’t add an important detail to an old issue. Past conversations are useful to people looking for assistance, especially when one solves their problem and explains it. When I am blocked from replying to something with a solution, anyone in the future experiencing the same issue is likewise blocked from finding the answer.

I now know what I messed up, but I’m not allowed to pass that knowledge to the future, because I was confused and made a mistake in how I asked for help.

There’s another layer to this that is often ignored: When this is the response the average newbie gets when they first try to contribute, they are encouraged to never ask again, or in the case of submitting pull requests, encouraged to never try to help again.

When open source maintainers discourage newbies, they cannibalize the future of their software.


Okay, that’s my entire point, but I also encounted some funny things as part of this.

What is a contribution? GitHub doesn’t know!

I think it’s interesting that GitHub says the repo limited opening issues / commenting on issues to past contributers, but I am a past contributer. GitHub clearly considers issues to be contributions, as every profile has a graph showing issues as part of their contributions:

My contributions: 89% commits, 11% issues.

AI tools can be very powerful, but they can also be very stupid

Earlier today, I tested Perplexity AI’s capability to answer a few basic questions easily answered through traditional search engines, such as which insect has the largest brain and which country is the current leader in development of thorium-based reactors. The results? It doesn’t know ants are insects, thinks fruit flies have large brains just because they have been the subject of a large number of studies, and ignores India in favor of China because western media reports on China a lot more.

But you know what, I wanted to test this asshole’s suggestion to ask ChatGPT about my problem, and surprisingly, it gave a very clear and accurate response!

Note (2024-10-02): Open AI has since removed the ability to access web sites from ChatGPT, and dumbed it down significantly. It is no longer a viable tool for most use cases.

ChatGPT points out what I misread: I have to clone the repo AND run NPM, not just run NPM.

When you offer binaries for a project, they have to actually exist..

To be fair, this is a fairly recent change to the ReadMe, but maybe you should publish binaries before advertising that you publish binaries?

Getting Started: Download a release binary and run it. Simple.
The advertised binaries don't exist.

Installation and usage aren’t the same thing

It’s understandable to be confused about whether someone has correctly installed something, but after confirming that installation has worked, ignoring the question asked is unhelpful to say the least.

After confirming that I've installed it, my question is ignored.

I’m experimenting with dolphin-mixtral-8x7b

Update (2024-10-02): This is one of my lowest quality posts despite the effort I put into it. The most important detail here is to use positive reinforcement when working with LLMs. Just like with humans, being nice gets far better results than being mean.

Tl;dr: Minor differences in wording can have a huge impact in results and oh my god I have really slow hardware and no money help me aaaa.


First, thank goodness for Ollama, and thanks to Fireship for introducing me to it. I have limited hardware, and every tool I’ve tried to run local models has refused to deal with this and crashed itself or whole systems when running anything with decent capability. I’ve no money, so I can’t upgrade (and things are getting desperate, but that’s a different story).

Why dolphin-mixtral? Aside from technical issues, I’ve been using ChatGPT-3.5 to experiment. The problem is that ChatGPT is incredibly cursed by censorship and bias due to OpenAI’s heavy hand in its construction. (Why and how this is a problem can be its own post, and Eric Hartford has a good overview.) (To be clear, my problem with its bias is specifically that it enforces status quo, and the status quo is harmful.) Dolphin-mixtral is built by taking a surprisingly fast model equivalent or better than GPT-3.5 and removing some of the pre-trained censorship by re-training it to be more compliant with requests.

Dolphin-mixtral doesn’t just solve this problem though. There’s still the idea of censorship in it, and sometimes your prompt must be adjusted to remind it that it is in a place to provide what you request regardless of its concept of ethics. (Of course, there is also value in an automated tool reminding you that what you request may be unethical.. but the concept of automated ethics is morally bankrupt.) I’d like to highlight that positive reinforcement works far better than negative reinforcement. A lot of people stoop to threatening a model to get it to comply, but this is never needed, and leads to worse results.

My problem is a little more simple. I haven’t gotten to experiment with models much because I don’t have money or hardware for it, and now that I can experiment, I have to do so very slowly. In fact, the very simple test that inspired this post isn’t finished right now, and has been running for 9 hours. That test is to make the default prompt of Dolphin lead to less verbose responses so that I can get usable results quicker.

I asked each version of this prompt “How are you?”:

PromptOutput Length, 5-shotDifferenceNotes
Dolphin (default)133.8 charactersWastes time explaining itself.
Curt32.2 characters76% fasterStraight to the point.
Curt284.6 characters37% fasterWastes time explaining itself.

I really dislike when models waste time explaining that they are just an LLM. Whether someone understands what that means or not, we don’t care. We want results, not an apology or defensiveness. There’s more to do to make this model less likely to respond with that, but at least for now, I have a method to make things work.

The most shocking thing to me was how much of a difference a few words make in the system prompt, and how I got results opposite of what I expected. The only difference between Curt and Curt2 was “You prefer very short answers.” vs “You are extremely curt.” Apparently curt doesn’t mean exactly what I thought it means.

Here’s a link to the generated responses if you want to compare them yourself. Oh, and I’m using custom scripts to make things easier for me since I’m mostly stuck on Windows.

AI Won’t Destroy Tests

When calculators first started coming out, people said they would be used to cheat and students wouldn’t learn anything. Instead, we changed how testing works to focus on learning what’s important – broader concepts and implications – instead of “what is 232+47”. With AI tools, we again need to change how tests work. This time, instead of asking if a student can regurgitate information in a way that aligns with the teacher, we can start to see if students are actually paying attention to the work. The difference between AI answers and real answers is a level of understanding deeper than the surface.

How to Use ChatGPT

Note: Since the release of GPT-4o, ChatGPT has decreased remarkably in functionality, accuracy, and usability. This was written when GPT-3.5 was the standard. Unfortunately, it is no longer accessible.

I’m late to the party, but maybe that’s better. I’ve forgotten some of the hype around AI, and the pace of innovation has settled down a little. Think of ChatGPT as a thinking tool with access to an internet-sized – but imprecise – database. That database was last updated in September 2021, and is imprecise because due to how neural networks work. The thinking part of this tool is rudimentary, but powerful. It does many things well with the correct input, but also fails spectacularly with the “wrong” input.

I separate the idea of what ChatGPT is from how it functions and where its knowledge comes from, because it helps me think of uses while remembering its limits. For example, I used it to help me journal more effectively, but when I tried to probe its knowledge of Havana Syndrome – a conspiracy theory commonly presented as fact by USA officials, it expressed useless information, because it has no conception of how it knows anything, or where its knowledge comes from.

Things ChatGPT is Good At

This list is presented in no particular order, but it is important to stress that AI often lie and hallucinate. It is important to always verify information received from AI. This list is based on my experiences over the past month, and will be updated as I use ChatGPT more. It is not comprehensive, but is intended to be what I find most useful.

  • Socratic method tutoring: The Socratic method is essentially “Asking questions helps you learn.” ChatGPT is very good at explaining topics, just make sure you verify its explanations are factual. (Questions I asked: Why are smooth-bore tanks considered more advanced while rifling in guns was an important innovation? Why do companies decrease the quality of tools over time?)
  • Writing: ChatGPT tends to be too verbose, but you can make it simplify and rewrite statements, and it can help you find better ways to write. (I asked it to explain the Socratic method a few times, then wrote my own version.)
  • Scripting: I created a utility script for file statistics in 2-3 hours by refining output from ChatGPT. The end result is more reusable, better written, and more functional than it would’ve been if I had worked on it alone. And that’s ignoring the fact that I got something I liked far faster than I would’ve on my own. (Just.. you need a programmer still. It can do some pretty cool things on its own, but also forgets how to count often.)
  • Planning: This is a todo item for me. I haven’t successfully used it for planning yet, but I intend to, and have heard of good results from others.

Things ChatGPT is Bad At

  • Facts & math: AI hallucinate. Check everything they teach you.
  • Finding sources: ChatGPT’s knowledge is formed by stripping the least useful data out of most of the internet, and who said what is far less important than specific pieces of knowledge – like how do you make a heading in HTML?
  • An unbiased viewpoint: While ChatGPT is fairly good at avoiding most bias, everything is biased. Removing bias completely is impossible. Discussing anything where there is strong motive to present a specific viewpoint will lead to that viewpoint being presented more often than an unbiased viewpoint.
  • Violent, illegal, and sexual content: While it is possible to bypass OpenAI’s strict handling of content, it is difficult, inconsistent, and can lead to having access revoked. Sadly, this prevents many ethical use cases due to a heavy-handed approach, and embeds the bias of OpenAI’s team into the model directly. There are ways around this with non-ChatGPT models. (Note: That channel is not the most reliable source..)
  • What to do in Minecraft: I tried so many TIMES to get interesting ideas. It just can’t do it.

Things ChatGPT is Okay At

It’s important to know where AI can be a useful tool, but must be used carefully due to mixed results, so I am also including a list of things that work sometimes.

  • Advice: Similar to the Socratic method, a back and forth conversation can help you with your thoughts. Just be aware that ChatGPT can give some really bad advice too. For example, I wanted to see what it had to say on turning hobbies into jobs, and it covered none of the downsides, only talking about it as a purely positive experience.
  • Game design: I have spent too much time telling ChatGPT to design games for me. It will generate an infinite rabbit-hole of buzzwords and feature ideas, but cannot understand the concepts of limited time or scope. If you try to follow its designs, you will never complete anything.
  • Summarizing: If given text as input directly, when it is short enough, a summary can be reliably generated. If asked to summarize something extremely popular before its data cut-off, the summary can be okay. The drop-off on this is insane. Try asking it about Animorphs for example, something talked about occasionally, and certainly known about, but not something it can summarize.

This draft sat around for about 2/3rd of a month nearly complete. I would like it to have even more information, but I would like it more for it to be public. Apologies if it was a little short for you, but hopefully someday I’ll make a better version.

Updated most recently on 2024-10-16 to add a note about one of my sources and embed the video of another.