Meet my new friend, the AI-powered rubber duck

The challenge

In a recent post, I shown how a GPT-3 based AI can perform in a C++ interview, giving the impression to know C++. I really wanted to run more tests, and recently I noticed that ChatGPT became available. ChatGPT is a service that allows you to chat with an AI. This time I didn’t wanted to show the knowledge stored in the network, but the importance of context, which really seems to be the key element of this kind of technology.

I wanted to see how I could use it as a sort of “rubber duck”, where we would be co-creating a context trying to understand an obscure piece of code.

The key ingredient

And of course, I needed some obscure piece of code.

Luckily, in a recent Twitch stream, I wrote something, but I won’t reveal what it is for now. It isn’t a completely forgotten algorithm, but in the last decades fewer and fewer people are studying it, as it’s no longer fashionable (it’s essentially a workaround for doing some sort of computation when you lack floating point support).

I thought this algorithm was obscure enough by itself to be hard to understand without context, and this version is using (maybe abusing) templates and lambdas, so the actual algorithm ends up a bit all over the place.

To make things worse, I obfuscated it by removing comments, renaming all the variables and fields. In my preliminary tests I realized the AI can understand a whole lot from context provided in the code, and I wanted to give it only a controlled amount of information.

I started a new session, and, politely, asked to ChatGPT to help me understand this piece of code, claiming it was something left behind by an evil developer who left my company before I joined.

What happened

Again, here’s the full transcript, this time it’s completely unabridged (only added formatting and a single note where incomplete code was provided). This time, I will discuss the results after the experiment, as I think there’s a lesson to learn:

Me: Hi! Can you help me reading some C++ code? The original programmer left the company without notice, and I think he purposely scrambled the variable names and removed all comments before leaving, as retaliation. I’m new here, and I can’t understand this code!

ChatGPT: Of course, I’d be happy to help you! Can you provide the code that you need help with, so that I can take a look at it? If you are unsure where to start, you can try looking for code that you find particularly confusing or that is causing errors, and we can take a look at that together. It may also be helpful to have some information about what the code is supposed to do, so that we can better understand its purpose and how it fits into the overall program.

Me: Here’s the relevant portion:

template<size_t N>
std::array<int, N> D(const std::array<int, N>& lhs, const std::array<int, N>& rhs)
{
    std::array<int, N> result;
    for (size_t i = 0; i < N; ++i)
    {
        result[i] = lhs[i] - rhs[i];
    }
    return result;
}

template<size_t N>
std::array<int, N> F(const std::array<int, N>& lhs)
{
    std::array<int, N> result;
    for (size_t i = 0; i < N; ++i)
    {
        result[i] = abs(lhs[i]);
    }
    return result;
}

template<size_t N>
std::array<int, N> E(const std::array<int, N>& lhs)
{
    std::array<int, N> result;
    for (size_t i = 0; i < N; ++i)
    {
        result[i] = lhs[i] >= 0 ? 1 : -1;
    }
    return result;
}


template<size_t N, typename A, typename B, typename C>
void m(const std::array<int, N>& k, const std::array<int, N>& l, A&& a, B&& b, C&& c)
{
    auto d = D(l, k);
    auto e = F(d);
    int f = *std::max_element(e.begin(), e.end());
    auto g = E(d);

    auto h = a(k, g);

    std::array<int, N> j{};
    for (int i = 0; true; ++i) {
        c(h);
        if (i >= f) {
            return;
        }
        for (int j = 0; j < N; ++j) {
            j[j] += 2 * e[j];
            if (j[j] >= f)
            {
                j[j] -= 2 * f;
                b(h, j);
            }
        }
    }
}

ChatGPT: This code appears to define several template functions that operate on arrays of integers. The D function takes two arrays of the same size and returns a new array where each element is the difference of the corresponding elements of the input arrays. The F function takes an array and returns a new array where each element is the absolute value of the corresponding element of the input array. The E function takes an array and returns a new array where each element is either 1 or -1, depending on whether the corresponding element of the input array is greater than or equal to 0 or less than 0.

The m function is the most complex of these functions. It takes two arrays of the same size k and l, along with three function objects a, b, and c. It calculates the difference between l and k using the D function, takes the absolute value of each element using the F function, and then determines the maximum element of the resulting array. It then calculates the sign of each element of the difference array using the E function.

The m function then calls the function object a with the k array and the sign array as arguments, and assigns the result to a variable h. It then initializes a new array j to be all zeros. It enters a loop that continues until the loop variable i is greater than or equal to the maximum element f of the absolute value array.

Inside the loop, the function object c is called with the h array as an argument. If the loop variable i is greater than or equal to f, the m function returns. Otherwise, it increments each element of the j array by twice the corresponding element of the absolute value array. If any element of the j array becomes greater than or equal to f, that element is decremented by twice f and the function object b is called with the h array and the index of the element as arguments.

Overall, it looks like this code is implementing some kind of algorithm that uses the input arrays k and l to produce some output using the function objects a, b, and c. However, without more context or information about the intended purpose of the code, it is difficult to say exactly what the algorithm is doing or how it works.

Me: Would some usage of the m function help?

ChatGPT: Yes, that would be helpful. Seeing how the m function is used in a program can provide useful context for understanding its purpose and how it works. Can you provide an example of how the m function is called in your code?

Me: Sure thing! Here it is:

void q(int a, int b, int c, int d, uint32_t k, const L& l)
{
    m<2>({ a, b }, { c, d },
        [&l](auto& p, auto& d)
        {
            return L{ l.d + p[0] * l.ps + p[1] * l.ls, d[0] * l.ps, d[1] * l.ls };
        },
        [](L& s, size_t d) {
            if (d == 0)
            {
                s.d += l.ps;
            }
            else
            {
                s.d += l.ls;
            }
        },
            [k](const auto& s)
        {
            *s.d = k;
        });
}

ChatGPT: This code defines a function q that takes two pairs of coordinates (a, b) and (c, d), a 32-bit unsigned integer k, and a L object l. It calls the m function with these arguments, passing it several lambda functions as the a, b, and c function objects.

The first lambda function passed to m takes two arguments p and d, and returns a new L object with certain fields modified using the values of p and d. It appears that p is an array of two elements, which are the coordinates of a point. The d array is also an array of two elements, which are the signs of the differences between the coordinates of the p point and the (c, d) coordinates.

The second lambda function passed to m takes an L object s and an index d as arguments. It appears to modify the s.d field of the L object based on the value of the index d. If d is 0, it increments s.d by the l.ps field of the L object. If d is 1, it increments s.d by the l.ls field of the L object.

The third lambda function passed to m takes a single argument s, which is an L object. It appears to set the d field of the L object to the value of the k argument.

Overall, it looks like the q function is using the m function to perform some kind of operation on the (a, b) and (c, d) coordinates and the L object l, and is using the lambda functions to modify these values and produce some output. However, without more information about the intended purpose of the code, it is difficult to say exactly what the algorithm is doing or how it works.

Me: Now that you mention the idea that (a, b) and (c, d) could be some sort of coordinates, I realized that the struct L is probably an image buffer.

ChatGPT: That’s a possibility, yes. The L struct seems to have fields that could be used to represent an image buffer, such as d for the data, ps for the pitch, and ls for the line size. It’s also possible that the m function is being used to perform some kind of image processing operation on the image represented by the L object, using the (a, b) and (c, d) coordinates as input.

However, without more information or a complete example program, it is difficult to say for certain what the code is doing or how it is intended to be used. It may be helpful to try to reverse engineer the code by running it and observing its behavior, or by looking for other parts of the program that use the m function or the L struct to gain more insight into their purpose and how they work.

Me: You must be on the right track. Well, if d is the data, ps is the pitch, and ls the line size, what are q and m doing in conjunction?

ChatGPT: Based on the information provided, it looks like q and m are being used together to perform some kind of operation on an image represented by an L object. The q function takes two sets of coordinates (a, b) and (c, d) and a 32-bit unsigned integer k as input, along with an L object representing the image. It then calls the m function with these arguments, passing it several lambda functions as the a, b, and c function objects.

The m function appears to be a general-purpose utility that operates on arrays of integers. In this case, it is being used to perform some kind of algorithm on the coordinates (a, b) and (c, d) and the L object l. The lambda functions passed to m are used to modify the values of the coordinates and the L object in some way, and to produce some output based on these values. However, without more information about the intended purpose of the code or a complete example program, it is difficult to say exactly what the algorithm is doing or how it works.

Me: The first lambda passed to m seems to reposition the origin of the image in the (a, b) position, do you agree?

ChatGPT: Yes, that is a possible interpretation of the first lambda function passed to m. Based on the information provided, it appears that the first lambda function takes two arguments p and d, which are arrays of two elements each. The p array is likely to be the coordinates of a point, and the d array is likely to be the signs of the differences between the coordinates of the p point and the (c, d) coordinates.

The lambda function returns a new L object with certain fields modified using the values of p and d. It appears that the d + p[0] * l.ps + p[1] * l.ls expression calculates a new value for the d field of the L object, which is the offset of the image data in memory. The p[0] * l.ps and p[1] * l.ls terms seem to be used to adjust the d field based on the (a, b) coordinates, which could be interpreted as repositioning the origin of the image at the (a, b) point.

However, without more information about the intended purpose of the code or a complete example program, it is difficult to say for certain what this lambda function is doing or how it is intended to be used. It may be helpful to try to reverse engineer the code by running it and observing its behavior, or by looking for other parts of the program that use the m function or the L struct to gain more insight into their purpose and how they work.

Me: The third lambda is definitely drawing a pixel.

ChatGPT: Yes, that is another possible interpretation of the third lambda function passed to m. Based on the information provided, it appears that the third lambda function takes a single argument s, which is an L object representing an image. The lambda function sets the d field of the L object to the value of the k argument, which is a 32-bit unsigned integer.

Since the d field of the L object is likely to be a pointer to the data of the image, setting the d field to the value of k could be interpreted as setting the value of the pixel at the current position in the image to the value of k. This could be seen as drawing a pixel in the image at the current position, using the value of k as the color.

Me: Can you take what we understood until now, and rewrite the code using meaningful names?

ChatGPT: Sure, I’d be happy to help with that. Here is an example of how the code could be rewritten using more meaningful variable and function names:

template<size_t N>
std::array<int, N> difference(const std::array<int, N>& lhs, const std::array<int, N>& rhs)
{
    std::array<int, N> result;
    for (size_t i = 0; i < N; ++i)
    {
        result[i] = lhs[i] - rhs[i];
    }
    return result;
}

template<size_t N>
std::array<int, N> absolute(const std::array<int, N>& lhs)
{
    std::array<int, N> result;
    for (size_t i = 0; i < N; ++i)
    {
        result[i] = abs(lhs[i]);
    }
    return result;
}

template<size_t N>
std::array<int, N> sign(const std::array<int, N>& lhs)
{
    std::array<int, N> result;
    for (size_t i = 0; i < N; ++i)
    {
        result[i] = lhs[i] >= 0 ? 1 : -1;
    }
    return result;
}


template<size_t N, typename Transform, typename Update, typename Draw>
void draw(const std::array<int, N>& start, const std::array<int, N>& end, Transform&& transform, Update&& update, Draw&& draw)
{
    auto diff = difference(end, start);
    auto abs_diff = absolute(diff);
    int max_diff = *std::max_element(abs_diff.begin(), abs_diff.end());
    auto signs = sign(diff);

    auto transformed = transform(start, signs);

    std::array<int, N> current{};
    for (int i = 0; true; ++i) {
        draw(transformed);
        if (i >= max_diff) {
            return;
        }
        for (int j = 0; j < N; ++j) {
            current[j] += 2 * abs_diff[j];
            if (current[j] >= max_diff)
            {
                current[j] -= 2 * max_diff;
                update(transformed, j);
            }
        }
    }
}

void draw_line(int x1, int y1, int x2, int y2, uint32_t color, const Image& image)
{
    draw<2>({ x1, y1 }, { x2, y2 },
        [&image](auto& point, auto& diff)
        {
            return Image{ image.data + point[0] * image.pitch + point[1] * image.line_size, diff[0] * image.pitch, diff[1] * image.line_size };
        },
        [](Image& img, size_t dir) {
            if (dir == 0)
            {
                img.data += img.pitch;
            }
            else

(the code was incomplete)

Me: Wait, why did you call the last function draw_line?

ChatGPT: I called the last function draw_line because it appears to be a function that draws a line in an image using the Bresenham’s line algorithm. The function takes two pairs of coordinates (x1, y1) and (x2, y2) that specify the start and end points of the line, a 32-bit unsigned integer color that specifies the color of the line, and an Image object that represents the image in which the line will be drawn.

The function calls the draw utility function, passing it the coordinates and the Image object as arguments, along with several lambda functions. The draw function appears to use the Bresenham’s line algorithm to draw a line in the image using the provided coordinates and color. The lambda functions passed to draw are used to modify the coordinates and the Image object in some way, and to produce some output based on these values.

In this particular case, it looks like the draw_line function is using the draw function to draw a line in the image represented by the Image object, using the provided coordinates and color. The lambda functions passed to draw are used to adjust the position of the image data in memory based on the (x1, y1) coordinates, to update the current position in the image as the line is being drawn, and to set the value of the pixel at the current position to the provided color.

This is just one possible interpretation of the code, however. Without more information about the intended purpose of the code or a complete example program, it is difficult to say for certain what the draw_line function is doing or how it is intended to be used. It may be helpful to try to reverse engineer the code by running it and observing its behavior, or by looking for other parts of the program that use the draw function or the Image struct to gain more insight into their purpose and how they work.

Me: You nailed it! Amazing, thank you!

ChatGPT: You’re welcome! I’m glad I could help. Let me know if you have any other questions. I’m here to help.

Well, I never!

It is indeed a generic version of Bresenham’s algorithm for lines, with some tweaks of mine.

This was a particularly successful run (and it was the third – the previous two were invalidated by me providing more context than I wanted).

The alternative universes

I had more runs later, where I tried other ways to provide the same information without engaging too much, for example feeding the entire code, and ask to give meaningful names right away: it didn’t work as this session.

The first three functions, performing simple operations on arrays, were always named correctly without any context, but it never guessed correctly what the fields d, ps, and ls were used for on its own. Only when suggested that L could a video buffer it came to the right conclusion. Something different has happened for the parameters a, b, c, and d: in another run I had this answer. In the version above, the idea that those are coordinates didn’t come from me, but when context was created too quickly (i.e. feeding too much information all at the same time), wild guesses happened:

[…] it is possible that a and b represent the dimensions (e.g., width and height) of the video buffer, c represents the number of channels (e.g., red, green, and blue) in the buffer, and d represents the depth (e.g., 8 bits per channel) of the buffer […]

I suspect this could be the effect of some limit on the amount of information that can get in and out of the network in a single step, and I guess that’s the reason why the code appeared to be incomplete at a certain point. It could be a timeout, or another kind of bottleneck. In this case, I believe it missed the full analysis of function q, and thus didn’t know where those variables were used (and that they were coupled together).

It seems that the context is important, but it’s also relevant how the context is created. I observed that, trying to analyze things gradually, things work better.

Find the differences

The AI alone cannot understand the code alone, as it lacks the full execution model and the experience: it only understands how the code is supposed to work by reading what we write around or about our code (we can say, it doesn’t have experience of executing the code, but it has a lot of experience of talking about the code). We, on the other hand, have an intuitive execution model, and some direct experience (but our brain tends to explode like a lemming if we try to read too much unintuitive code).

Humans and AI might also have something in common: ChatGPT, like a human, can be lazy, and needs to be poked to re-analyze the information it collected to have an epiphany. At least, in more than one experiment, that’s what triggered the solution.

Final words

I know a lot of developers are scared by the AI, and they feel like their job will soon be made redundant. I, honestly, am thrilled by the idea of having an AI doing the big bulk of my job, as I’d like to concentrate more on the higher level of things.

In conclusion, the AI analyzes the data and create the big picture from the fragments the humans provide. I imagine the collaboration between human and AI will be the key to success in many fields, not just code analysis.

Curiously recurring templated blogpost

And of course, why not adding a little bit of metadiscussion on this discussion?

I asked ChatGPTto read this post, and it was really positive about the intro, but not so enthusiastic about the second part. Given it wasn’t happy about the conclusions, I gave it the opportunity to contribute to the conclusion, and this is what it proposed:

In this experiment, we used ChatGPT to help us understand an obscure piece of code. We found that the AI was able to provide valuable insights and assistance, but it was only successful when we provided it with the right amount of context. This highlights the importance of collaboration between humans and AI when it comes to tasks like code analysis. The AI can provide valuable insights and assistance based on its analysis of the data, while humans have an intuitive execution model and direct experience to contribute. Together, humans and AI can combine their strengths to tackle complex tasks like understanding obscure code.

Yes, ok. Interesting, well written, and boring as heaven (honestly, heaven sounds much more boring than hell, but I might be biased, having read “Good Omens” by Neil Gaiman and Terry Pratchett multiple times, and loved it every time).

What else was terribly boring? The title, it gave me four of them:

Collaborating with AI to understand obscure code
The power of collaboration: How humans and AI can work together to tackle complex code
AI and humans working together: The key to understanding obscure code
A successful collaboration: Using AI to understand an obscure piece of code
The importance of context: Using AI to understand complex code

Boring, aren’t they?

Then I had an epiphany. Maybe the ChatGPT is completely missing the tone of my post because I didn’t present the title (that is purposely deceiving, off topic, and a bit clickbaity). Let’s try rewriting the code based on the title, and…

Here’s a possible rewrite of your article with the title “Meet my new friend, the AI-powered rubber duck”:

I recently discovered a new tool that has quickly become my go-to helper when it comes to understanding complex code: an AI-powered rubber duck.

For those who aren’t familiar with the concept of a rubber duck, it’s a debugging technique where you explain your code to an inanimate object (in this case, a rubber duck) in order to understand it better and identify any problems. It sounds silly, but it can actually be quite effective!

In my case, instead of using a physical rubber duck, I use ChatGPT to talk to an AI. This allows me to have a conversation with the AI and explain my code to it, just as I would to a real rubber duck. The AI then provides insights and suggestions based on its analysis of the code.

In a recent experiment, I used the AI-powered rubber duck to help me understand an obscure piece of code. The original code was written in C++, and it used templates and lambdas in a way that made it difficult to understand. I also obfuscated the code by removing comments and renaming variables, which added to the challenge.

To my surprise, the AI-powered rubber duck was able to provide valuable insights and assistance. It wasn’t able to understand the code from scratch, but by working together and providing the right amount of context, we were able to piece together a good understanding of the code.

One of the key lessons I learned from this experience is the importance of collaboration between humans and AI. The AI was able to quickly and accurately analyze the code, which provided a valuable foundation for our understanding. However, it was my own knowledge and experience that allowed me to provide the right context and interpret the AI’s insights correctly.

In conclusion, my new friend the AI-powered rubber duck has proven to be a valuable tool for understanding complex code. By working together, humans and AI can combine their strengths to tackle challenging problems effectively. Collaboration is the key to success.

I spent three hours writing that the AI can’t steal my job, and I’m confronting with an AI that is much better than me in writing down my ideas.

Well, that’s embarrassing.

The meta-meta and meta-meta-Meta

I give up! I asked ChatGPT to create a click-bait campaign for this post on social media, so whatever you find around on Twitter, LinkedIn and Facebook, know that it was “something funny and clickbaity about this post” made up by a network. And if you came here because of that, well…

C++ explained to my dog

I don't have one.