My LLM Journey as a Software Engineer Exploring a New Domain

Over the holidays when I had some time off, I pondered the thought of how to make some passive income. In this day and age of AI and all the hype, I had been thinking that the world is my oyster.

With the threat of AI taking over the world, especially in my day to day as a software engineer, I should start bucking up my ideas and get involved.

So, what do I do?

AI is smart, right? It can blurt out code that I don’t have committed to memory. Furthermore, it can churn out mathematical algorithms way smarter than I could. It solves the conundrums I’m thinking about in a second. It’s my new sidekick; I can do anything!

So, why not try something new? There is some hype going around that a non-developer can spin up a fully-fledged app. They will be able to whip up complex APIs, have them secure, running efficiently, and have some persistent storage. The list goes on and on, but ultimately, they make money! 

I’ll put myself in their shoes and make the impossible possible!

Now, how do I get started?

Let’s Try Something New

I have no idea about trading, stocks, and crypto. With the backing of an LLM at my side and its fountain of knowledge, I’m surely on track to crack this industry.

As a newbie to the domain and the confidence of a superior AI to roll out smart trading algorithms, I’m like that hype that is being broadcasted: AI can do it, AI can do it for me. 

I started my research. I watched videos, read blogs, and got my head into the space. After all, I’m a developer, I can do this!

In a naive mindset, I’m thinking of all the possibilities. My primary thought centered around making a “trading bot” and letting it run and automatically trade. I would be making money while I slept with this newfangled LLM steering the way.

My thought process was that if a currency is worth 100 dollars, it could go up to 120 dollars. There’s a window there to gain 20 dollars if bought and sold at the right time.

With this idea in my mind, I start to hammer an LLM for a while. I was dreaming up scenarios and asking newbie questions.

I’m sold. This could be the key to an early retirement.

Let the Journey Begin

Again, I had to remind myself that some non dev is going to come along and use an LLM to take my job, as they will be able to whip this up in no time.

To trade, one needs some form of broker to be able to do this. Being in the UK is somewhat restricted compared to the US in terms of functionality and access to markets.

So, I set up an account with Coinbase. The first tricky part is accessing their API, and the second tricky part is being able to find the right API docs. It wasn’t easy digging through a whole bunch of resources that I was not familiar with. 

Looking around the advanced trading screen, I was overwhelmed with information. There were vast amounts of options, and the lovely candle stick graphs that in and of itself, were whole new topic to learn and research.

And thus, there were more YouTube videos and blogs to consume.

I then jumped to the complex API documentation. There were 100s of new words and industry terms. “It’s okay,” I said to myself, “I can use an LLM to guide me.”

I wanted to jump straight into some code and at least call something from their API.

This was first stumbling block, but the LLM came to the rescue. I asked it to give me an example using the Coinbase API to get candlestick data.

For a newbie, this would be very tough to figure out. In involved API keys and secrets, and the creation of a client with fields. It also needed the calling of the API with specific params, parsed into specific types (example: unix time stamps, floats, and strings).

The LLM provided a rough example that was using an incorrect and out-of-date client, but for the most part it was ok. After some tweaking, the fields matched up with the request and voila! I was getting the  candlestick data back.

Here’s an example of the candlestick chart data structure:

 [{
 	start: '1736201400',
 	low: '101683.99',
 	high: '101800.17',
 	open: '101762.41',
 	close: '101754.57',
 	volume: '13.58472385',
 },
 ...
 ]

I was getting back 1000s of results. Again, I had to do some research to figure what these actually meant, which I won’t go into here, but the main value is the close field.

Looking at the API, I realized that there were lots of endpoints that I needed to call. Luckily i found a Typescript library Coinbase that contained all the endpoints.

This library is not part of npm, so it required building it locally and adding it to your project. In doing so, it was apparent that none of the types were exported, and that meant I would have to create all the data transfer objects (DTOs) to call and consume the library. I did a quick-and-dirty fork of that repo and exported the types which can be found here.

Now, I was able to build the forked repo and simply run npm link and then in my project I could consume that by running the following: npm link @coinbase-samples/advanced-sdk-ts

This then exposed to the types so i could use things like: import { RESTClient } from '@coinbase-samples/advanced-sdk-ts';

And it also exposed this:

this.clientInstance = new RESTClient(
  process.env.COINBASE_API_KEY,
  process.env.COINBASE_API_SECRET,
);

Yikes! That’s a lot of faffing around to get this up and running; I think my job is going to be safe from AI taking over for some time yet with these shenanigans.

Say the AI newbie got around this issue and can call these APIs. We’re at a level playing field now.

Down to Business

So, we need a strategy. The videos I watched all seemed to mention RSI, the Relative Strength Index. So I started off with that.

In a nutshell, it takes a period of close values, default of 14 values, and calculates an index. When the index goes below 30, this is a sign of overselling and indicates the market price might start rising – bullish. If it goes above 70, then it’s a sign of overbuying and indicates the the price may start falling – bearish.

Sounds amazing! Buy when it’s 30 sell when it 70. Cha-ching!

I found out that it was pretty complicated to do this calculation, and I didn’t want to spend time doing it. Fortunately, there is a technical-indicators library to use, so I’ll run with that and expect that it’s solid, precise, bug free, fully tested, and works as it should.

When I used an LLM,  I gave it my prompt in the region of use that the library and function had to get the candles from the API, and I had another function to buy and sell when the set values were hit. It did as I asked and gave a long script to carry out these actions.

However, in reality, it didn’t really work too well. The call to the API was incorrect. The RSI function shifted the last element from the index and didn’t provide an index value.

After a short while of tweaking, I broke up the functionality into smaller components and separate files. The API was now being called and the RSI function was returning something that looked like an index.

In order for it to react to the market, I needed to poll the candle endpoint to get the latest values, and plug that into the RSI function. That way when any “buy” or “sell” signals arise I can take action. 

I didn’t want to use to a timer for this as it would get out of sync, so I set up a dynamic cron job to run each minute, get the candle data, then pass it in to the RSI manager. Below is the cron job that I set up:

trade() {  
	this.job = new CronJob(
        '* * * * *', // Cron expression for every minute
        async () => {
          const candles = await this.getCandles(
            'BTC-USD',
            'ONE_MINUTE' as Granularity,
            1, // Minutes ago
            1, // Limit
          );
          this.rsiManager.updateWithCandle(candles.candles[0]);
        },
        null,
        true)
}    

I created an rsiManager to handle the logic. This made it easier for the LLM to give me a complete file and could copy and paste as changes happen.

So, at this stage I could fire up the app and start calling the candle API, run it through the RSI, and get a sign to buy or sell.

I left it running for a while and monitored the logs, and saw some buys and sells. Great! I checked these against the Coinbase graph console to get an idea of what the candlestick looks like.

This isn’t the exact RSI index signal i got but it’s something similar to what I’m after:

This is great and I’m on track for an early retirement for sure!

After some time, I saw the RSI function was constantly buying. In real life, this would start to wipe out my finances. So, I threw this back to the LLM, telling it the issue and have it give me some new code. 

After some back and forth, the LLM was getting more confused. It was changing the code for no reason. It was forgetting to apply conditions that I told it to apply before. I started a new conversation a few times and that seemed to help.

Test Test Test

With the realization that the pattern of buying could easily wipe out my account let alone any profits, I decided to get the boilerplate setup to start creating some tests to ensure that the RSI was actually working as expected.

This proved pretty difficult as the test data itself had to simulate the peaks and troughs of the market. With a limited knowledge of the domain, I struggled to create any meaningful test data. The LLM would blurt out a series of numbers that looked pretty simple. But when you’re trying to get patterns for days and weeks, it became very complicated.

Was this a clear sign of me not knowing what was doing? 🙂

I moved on to just consuming the API and simulating buys and sells, adding conditions like when I’m in sell mode to not buy anything.

In addition to this, I got the LLM to start tracking and providing analysis of trades. This would give me data as seen below:

Cron job started.
{
  analysis: { rsi: 50.15, signal: 'HOLD', reason: 'Waiting for conditions' },
  currentPrice: 94312.93,
  time: 2025-01-11T14:14:00.246Z
}
{
  totalTrades: 0,
  winningTrades: 0,
  losingTrades: 0,
  breakEvenTrades: 0,
  winRate: 0,
  averageProfitPercent: 0,
  averageLossPercent: 0,
  largestWin: 0,
  largestLoss: 0,
  profitFactor: 0,
  totalProfitLoss: 0,
  maxDrawdown: 0,
  consecutiveWins: 0,
  consecutiveLosses: 0,
  maxConsecutiveWins: 0,
  maxConsecutiveLosses: 0
}

And of course I would assume the LLMs calculations would be perfect!

I was very tempted to get this fully up and running and actually making buys and sells, but I am glad I held back.

After running this for a few hours, it become evident that this strategy would not work.

The RSI would signal a buy. The price could fluctuate and drop by 5%. Then, when the market would pick up slightly and rise a couple of percentages, the signal to sell would happen. As you guessed it (or not), the price was already lower than the buy price and it would sell at a loss.

Again, I fed this back into the LLM and it endeavored to spit back the perfect working solutions. However, each time, the code became more complex and farther away from what I wanted.

I was time for me to step back and reevaluate that early retirement. 

I had the basic building block to start to build a bot. But I needed to learn a lot more about the trading market abyss.

Key Takeaways

Below are a few key takeaways that I learned from this LLM journey.

• Using an LLM in a new domain is not a silver bullet, because you don’t know what you don’t know.
• Given the complex nature of the domain and calculations required, you have to know what you need to do before you start. Otherwise, it’s possible to be led down the wrong path by the LLM’s confidence.
• It’s very difficult for the LLM to keep on track as requirements change. It was a battle copying and pasting code each iteration, especially when it would change functions.
• I may need to change approach and create a list for the prompt to manage requirements?
• If you have the domain knowledge, then I think using an LLM would be a good addition to your arsenal. 
• I found it very challenging to write meaningful tests to test out the algorithms.

Conclusion

The main aim of this project was for me to try and jump into the deep end in a new domain and create some software that could be used in the crypto trading market leveraging an LLM.

I hopped between ChatGPT, Claude, and CoPilot. I did not have a subscription for any, so once the free tier finished, I would jump on another LLM. Ideally I would choose only one to be my companion.

On the next attempt (if ever), I would approach this as I would with my day to day. I would create tests with data that I understand and simulate scenarios. Without an economics degree or years of data to crunch, this may be a taller order than I have time and energy for.

Overall, I had some fun with this. I learned a lot about crypto and trading and that the rabbit hole of it is massive. I have a long way to go before I get below just scratching the surface.

My naivety of the stock market would have lost me money. There are existing bots out there that claim 20% returns, but where’s the fun in that.

So, will I ever be a crypto millionaire? Well, that’s yet to be seen.