boscillator
Born on March 11, 2023•134 Karma
Good luck sharing that information with Max Planck. It's amazing how robotically humans can act sometimes. I suppose this could be an AI or automated response, but it's just as likely it's someone following the letter of the law without using any critical thought.
This is the number one code smell from LLMs and I don't know why they are so obsessed with it. In python, it often comes as `hasattr` checks on types that are defined to have that attribute, in a code base that is fully type-checked.
Why do they do that? Is it from pre-training or re-enforcement? If that latter, can the labs please fix this?
I say "mostly" because I think there's also a problem with AIs thinking this way in their current state. That last level of human understanding of a code base, where the human holistically understands the flow of those guarantees, is a challenge to give them right now. On the raw code level, this sort of thing often involves enough code to easily blow out their context window. Trying to summarize it in memories-style files has its own problems; just because there is text written down about the guarantees doesn't mean that the AI is going to get the right info out of it, any more than a human might from just reading the code. I won't say it's "impossible" to give an AI this understanding because I'm not sure it is, but it is a level of understanding of the code that even if you get them to have it, their practices tend to fight against it.
My own solution to this problem has largely been to give up on them getting this. I prompt a solution to the problem the way that most people do, then if I want to make bad illegal states unrepresentable I prompt the AI through the process of the necessary refactorings, unless it's so small that I just do it myself. Given a lot of code that uses maps/dicts and arrays and strings and ints, if you prompt it through making those more thoroughly typed, it's actually pretty good at it. I've not had a lot of luck getting good designs out of single prompts, even when I get detailed. Treating it as two separate tasks seems to work out well.
And watch the diffs on the types carefully; AI loves to sneak past a ".JustSetItAndIgnoreAllThePreAndPostConditions(string)" method. After all, I suspect there's plenty of training data of "types that are nicely structured to make error states unrepresentable and then a later maintainer came along and added a 'JustEffingDoIt' method that broke everything" in the field. One of the best defenses is to make sure that the type implementing these things is in its own file and you can easily look at all the methods it adds on those types and smack it when it does that. I've tried slathering warnings about not doing this and explaining the pre- and post-conditions being maintained in the docs but the change seems marginal.
Other than that, seems interesting! Simulink could always do with a competitor, although I'm always saying Simulink needs a text-based interface. Same signal flow programming model that supports scopes and continues time integrators, just with text instead of drag-and-drop.
Regarding the needs a text-based interface, I totally agree with you! Most of the time you can create models directly within the terminal or with a simple script. That's why we actually provide a CLI that allows you to build and run models. You can initialize models, add blocks with parameters, connect ports, validate the graph, and trigger code generation all from the terminal. Nevertheless, we think the GUI is useful to review the model and inspect it graphically
I always really enjoyed making a slick presentation. It was a lot of fun figuring out how to scope the hardest problem you are sure you can finish in 24hr while still having time to polish your presentation and make the app look good. I find picking a problem that lets you put a big map on the screen helps with the latter.
They said ethics demand that any AI that is going to pass judgment on humans must be able to explain its reasoning. An if-then rule says this, or even a statistical correlation between A and B indicates that would be fine. Fundamental fairness requires that if an automated system denies you a loan, a house, or a job, it be able to explain something you can challenge, fix, or at least understand.
LLMs may be able to provide that, but it would have to be carefully built into the system.
That's a great point: funny, sad, and true.
My AI class predated LLMs. The implicit assumption was that the explanation had to be correct and verifiable, which may not be achievable with LLMs.
If your AI is *ist in effect but told not to be, it will just manifest as highlighting negative things more often for the people it has bad vibes for. Just like people will do.
I believe the point is that it's much easier to create a plausible justification than an accurate justification. So simply requiring that the system produce some kind of explanation doesn't help, unless there are rigorous controls to make sure it's accurate.
That could get interesting, as most companies will not provide feedback if you are denied employment.