Chewing Glass - Cronos

Chewing glass is what Solana developers do. Introducing the fifth episode in a new series on the Solana Podcast, Chewing Glass. Chase Barker (Developer Relations Lead at Solana Labs) talks shop with the most interesting builders in the Solana ecosystem. It’s for devs, by devs.

Today's guest is Cronos, an on-chain task scheduler that allows users to schedule instructions and winner of the recent Riptide Hackathon. 

00:38 - Introductions

01:25 - How they started                 

02:48 - How they met                     

04:26  - Who else is in Austin            

05:09  -  Cronos backstory                 

07:34 - How they started building tasks    

09:11 - TLDR: what is Cronos?              

13:33  - Winning the Riptide Hackathon               

15:40  - How cronos came to life           

18:12 - Building on solana and familiarity with other languages

20:16  - Learning curve with rust         

22:50 -  Nick’s  learning curve              

25:04 -  Advice on learning curve         

27:08 - What's missing in Solana         

29:20  - Advice to new developers on Solana

DISCLAIMER

The information on this podcast is provided for educational, informational, and entertainment purposes only, without any express or implied warranty of any kind, including warranties of accuracy, completeness, or fitness for any particular purpose.

The information contained in or provided from or through this podcast is not intended to be and does not constitute financial advice, investment advice, trading advice, or any other advice.

The information on this podcast is general in nature and is not specific to you, the user or anyone else. You should not make any decision, financial, investment, trading or otherwise, based on any of the information presented on this podcast without undertaking independent due diligence and consultation with a professional broker or financial advisor.

Chase (00:38):

Hey everybody. And welcome to Chewing Glass, the show where we talk to developers building in the Solana ecosystem. Today, we have Nick and Elias from Cronos, the recent winners of the Riptide Hackathon. Welcome guys. How's it going?

Elias (00:50):

It's going well.

Nick (00:51):

Yeah, great to be here.

Chase (00:52):

So let's start with you, Nick. What's your history? How'd you get into this whole thing?

Nick (00:56):

My background is basically, I worked on the payments team at Uber for about four years or so, helping build out the payroll system there. And so, was working on a lot of international banking integrations and just became very aware how broken the current legacy payment system is. Can't really even tell you the number of times I got woken up at 4:00 AM because some system failed somewhere and had to email a CSV file to some banker to push money through the system, it happens all the time.

Nick (01:28):

And so I heard about Solana and I had a light bulb moment really, where I realized that this thing is cheaper and faster and easier to use than any existing payment rails that I know about and so, I got quite excited about the potential for disruption there and this was all pre Solana Pay stuff. So, yeah, we dove in during the Ignition Hackathon.

Chase (01:52):

Oh, you did? Cool.

Nick (01:53):

Yeah.

Chase (01:53):

Yeah, I didn't realize that. I was actually, when you were talking, I was thinking in my head, I was like payments? I was like, wow, didn't even build anything on Solana Pay. So you guys were already rolling a little bit before that, so that's cool. Yeah. So Elias, how about you?

Elias (02:06):

I'm pretty fresh as far as experience in industry. I did have an internship in college as a data scientist in Argentina for a bit. I then transitioned into front-end development that following year and had been a front-end developer up until when I got into Solana development. But that's pretty much it.

Chase (02:23):

Cool. So, with all that said, you guys' backgrounds, how did you guys end up meeting each other? What's the story? Are you in the same place? You guys are in the same city? How did that work?

Nick (02:34):

We're both in Austin. Elias is a few minutes north of Austin. I'm kind of downtown and we meet up down here a few days a week. We basically met on Twitter last summer.

Elias (02:45):

Yeah. I was at Samsung before I met Nick. I was basically a site reliability engineer for this semiconductor facility here in Austin. I didn't love it. I absolutely hated it. So I was creating toy projects on GitHub just for front-end development purposes, just to better my skills, because I wasn't really progressing that well at Samsung. I was then tweeting about it and my philosophy was, well, what's the worst that could happen? Someone's going to see this and maybe look at my repo, who knows?

Elias (03:13):

Randomly I get a DM from this guy named Nick. He was like, "Hey, I like what you're doing. I looked at your GitHub. We're looking for front-end developers at this product studio that we have in Austin. I would love to grab a beer." And that's really where it started.

Chase (03:25):

Oh wow. That's awesome. I thought that stuff only happened in Web 3, but I guess it's happening outside of that too. So how did you find that tweet, Nick, in the first place?

Nick (03:36):

I don't remember, honestly. It's like scrolling the timeline, don't really remember what you saw 10 minutes ago. But, I think I saw Elias tweeting, maybe a GitHub link or something, saw he was a dev and I was just looking through his projects on GitHub. And, I found his resume actually and everything there was kind of focused around like Next.js and React, which we were doing a lot with at the time. And so, I figured sending a DM couldn't hurt and just kind of realized that he was based in Austin. I had just moved here, I think a week or so prior and we met up and grabbed a beer and just hit it off from there and have been working together since.

Chase (04:18):

Very cool. I think there's a couple other people in Austin. I believe Castle Finance is there. There's actually a pretty decent Solana builder presence at University of Texas at Austin.

Nick (04:30):

The [inaudible 00:04:30] team is here as well.

Chase (04:31):

Oh nice.

Elias (04:32):

I think BuffaLou is also here.

Chase (04:34):

BuffaLou? The famous BuffaLou is in Austin. Are we doxxing him over here?

Elias (04:37):

No, he's tweeted about it.

Chase (04:39):

I'm just kidding. So, that's pretty cool to hear the story about how you guys met. Now, let's dive into a little bit, go a little bit further. Where did this idea get birthed or what were you building? And start at Ignition and lead up to how the idea of Cronos came about.

Nick (04:55):

As we were saying before, we were looking at Solana initially from that perspective of payments and coming from the payments industry. And so we started in the Ignition Hackathon building an on-chain Venmo where users could send and receive invoices and pay those back on-chain. And then that kind of rolled into a token streaming service. And-

Chase (05:16):

Was that called Cronos or did it have a name at the point in time?

Nick (05:20):

Yeah, that was called Factor at the time. What we were specializing on was the use cases of subscription payments and payroll. And specifically we were trying to figure out how to schedule token transfers because it's kind of these inefficiencies in the vesting contract model where the sender has to lock up future payments up front into investing contract. So there's some inefficiency there. And the receiver has to go out of the way to claim from the vesting contract. So we thought if we could schedule token transfers, maybe that would be a better user experience.

Nick (05:52):

And we were working on that for a few months, got the whole system up and running. And then around February 1st, 2022, when mtnDAO was taking off, we realized that we could generalize that protocol from only supporting token transfers to being able to automate any arbitrary instruction. And from there it just took on a life of its own.

Chase (06:14):

So you guys were at mtnDAO?

Nick (06:16):

Yeah, I was at mtnDAO before it was mtnDAO. There was a version of it in 2021 called Mountain Compound. It was way smaller, but it was 14 of us, or so, just locked down in a house, trying to escape COVID, in Salt Lake City. And that was where I first met Edgar and Barrett. Barrett, at the time, was already working on Solana and Edgar and I were working on separate startups, but I think we both got the Solana pill during that time.

Chase (06:47):

Man. Wasn't expecting that one. That's a really cool story, actually. Those guys are involved in red-pilling a lot of people on to Solana, so I'm always happy to hear these stories. They just keep coming up randomly wherever I go.

Nick (06:58):

Yeah.

Chase (07:00):

Yeah, so that's awesome. So when you were building this payment stuff, the idea came around at mtnDAO, and that was, or right at the beginning-ish, I think, of Riptide. TJ was just on the show. That's when he started to talk about building out mtnPay. So you guys were like, "okay, we were doing payments, we just came up with this thing. We think we have solved a really big problem and we're going to build this out." Tell me a little bit more about that.

Nick (07:25):

Yeah, it started with just a proof of concept. So we just had this basic question of, can you even schedule arbitrary instructions on-chain? And how do you do that? So we started by building a basic Anchor program where users could create tasks and each task is a different account. And inside those accounts we would store serialized instruction data with a schedule.

Nick (07:50):

We basically had set up a separate off-chain bot process, also written in Rust, but using the RPC client. Which basically watched for task accounts and then would trigger transactions whenever the tasks came due. And we found that we could invoke those inner instructions as CPIs and that then unlocked this whole like, okay, we can schedule any arbitrary instruction.

Elias (08:15):

Yeah. I remember whenever he called me on our sync, I think it was on Monday because he built the proof of concept during the weekend. He told me, "you know, we have Factor and it's really cool, but imagine if we just generalized it to allow for any arbitrary instruction." And I was like, "Oh. Yeah, let's do that. That's a good idea."

Chase (08:33):

Yeah, I was really stoked. I remember seeing it the first time and I saw what it was and I was like, "Wow, people are really going to like this." By the way you guys are both technical founders. You both built out Cronos, correct?

Nick (08:45):

Yeah, correct.

Nick (08:46):

Mostly Nick, let me just... Mostly Nick.

Chase (08:49):

Actually, this is probably a good point to talk about what Cronos actually is officially. Like a TLDR for everybody watching. What you guys built and how it actually works at a high level?

Nick (09:01):

The basic concept is, it's just a keeper network for Solana. Every blockchain, at least that we're aware of right now, has this fundamental limitation and that's, you can't schedule transactions with a validator network and there's a few different reasons why that's the case. But it creates challenges for teams that have background jobs or tasks that they need to run just to make their programs work. And so, what Cronos is, is a keeper network to be able to facilitate that and service that. But the main difference is that we're kind of turning the Solana validators into the keepers for the system rather than relying on some external, off-chain, opaque bot network. And so that's required a lot of deep integration with the validator codebase in order to enable that.

Chase (09:48):

That is actually very, very cool. I wasn't officially, 100% certain how it worked. So you're using the validators as the keeper network to run these jobs on the network?

Nick (09:58):

Yeah, exactly. Our v1, proof of concept version was not integrated into the validator network. Hadn't even had that idea at the time, really.

Elias (10:08):

I didn't even know we could do that, knowing that we can just build a plugin for validators. Pretty cool.

Nick (10:13):

Yeah, it was around the same time we were building that initial bot that we started seeing some tweets about the account's DB plugin framework. And that has since been renamed to Geyser plugin framework and we just realized that there was all these scaling problems when you rely on these off-chain bots and that they have to submit transactions through the RPC network. And that can take up a whole bunch of bandwidth and you have to compete with other traffic to get those transactions through. And we realized there was this interface that Solana was providing, and the Geyser plugin framework, that we could actually spawn transactions from there. And it was much more efficient and made the system a lot more reliable. And so we basically copy pasted our bot code into the Geyser plugin framework and it mostly just worked out of the box.

Chase (11:01):

Oh wow. And that's quite unusual. For things that just work. So did you guys actually have to work with the validator community or did you guys have ever set up or run a validator? What's your knowledge there?

Nick (11:12):

Yeah, we have a few nodes that we got through the Solana server program, which that is a very useful program, if there's anyone that's looking to set up a node on Solana. And we have some servers running on DevNet and Testnet right now, that we're using to stress test the system. But yeah, we've been reaching out to all the node operators we can to talk with them and we're looking to get this thing rolled out on DevNet and Testnet quite soon. And actually, by the time this is published, it should be out on DevNet and Testnet, and we'll have quite a few integrations going on those networks.

Chase (11:49):

Yeah. I'm not going to lie. So like just leading in the sense, congratulations, you guys won the Riptide Hackathon. This was super incredible. And for me personally, I was so insanely excited to see some tooling win because this is just... developers need this tooling and to see that people watching a hackathon and a lot of these other, in the past, DeFi protocols, which are amazing out there, winning, but to see developer tooling take the grand prize, says a lot about what you guys had built and what the judges thought of it. So, that's quite amazing. So congratulations. But tell me, what was that like? Were you guys, have any idea, any expectations? Like what was your thoughts going through all that?

Nick (12:33):

Man. Yeah, there was a lot going on at the time, even, even outside the Riptide Hackathon. It was quite a journey, I think, to get here. Cronos was what we wish we had when we were building Factor. We came upon the idea for Cronos because we were trying to build Factor, this scheduled token transfer service. And we're like, "how do you schedule a timer on-chain?" And then we found out you couldn't schedule a timer on-chain, there just isn't a way. So we were talking to some other teams and I think it was, he goes by DoctorBlocks, at Switchboard. He described for us what a Crank function was and how they were running their automations. And from there we just started pulling on that thread and realized that here was all this dev tooling that was missing that we could build out and just started running with it.

Chase (13:22):

Were you expecting to win the grand prize of the hackathon? How did you react whenever you actually found out that you guys had won that thing? Was there-

Nick (13:30):

I didn't know we were going to win. We had been getting tips from a few people that we were on these ever shorter shortlists, but we didn't know until the moment of, that the blog post went out, someone sent it to me and then a moment later, Twitter started blowing up. And from there it was just a flood of inbound messages coming in from all directions. And last few weeks have been a lot of dealing with that.

Elias (13:57):

Yeah. A lot of dealing about knowing what is spam and what isn't from people it's pretty difficult to do.

Chase (14:03):

So did you guys celebrate, did you guys go out for beers? Like you did the first time you met? Did you do anything?

Elias (14:08):

Sure did.

Nick (14:09):

Yeah.

Chase (14:11):

That's awesome, man. Like I said, it's really great to see some developer tooling win and that value in that. Whenever I started at Solana Labs, like a year ago, there was no developer tooling out there. This was like... Then comes Armani and then here's Anchor. And then now we have all these indexers and then now we have Cronos and they just keep piling on. And eventually we're going to reach a place where every little narrow gap is covered and developers are going to be able to just jump in and do all the things that they could do in Web 2, in Web 3 and it's going to be a huge game changer for everybody. Not quite there, or we're pretty far off from there, I would say. Every tool like this really, really matters.

Elias (14:51):

To piggyback off that, the most exciting part about this job is not only building it and dealing with really interesting engineering problems, but knowing the impact that it will have to developers and how empowering it is to allow them to automate things on-chain. That's a pretty wild idea. So I'm really excited for that.

Chase (15:09):

Yeah and I think that's why a lot of engineers get into building out developer tooling instead of products because they're engineers themselves. And they're like, "man, if I was like on the other end of this and somebody built this tool, I'd be so stoked." And how many people that outwardly impacts is probably just a really incredible feeling. And it's just really awesome. So sorry, Factor, but I'm glad that Cronos ended up winning. By the way, is Factor just kind of sitting on a shelf somewhere right now, never to be reopened again?

Nick (15:38):

Yeah. We, we kind of just rolled Factor into Cronos. Actually the Twitter account is the same Twitter. We just changed the name now.

Chase (15:46):

Nice. Okay. So it's dead.

Nick (15:48):

Yeah.

Elias (15:49):

Dead, but very much alive.

Nick (15:51):

Yeah.

Chase (15:52):

Yeah. If Cronos would've never came alive, you would've been sitting at the mtnDAO with TJ, directly competing against each other. So, that's awesome. So basically two of these projects were the winning of the payments track and then the grand champion of Riptide. They both came out of mtnDAO. Every time I hear about mtnDAO and we talk about this, it's one more reason why understanding how incredible it was out there and how many builders were out there building really cool stuff.

Elias (16:20):

Yeah, the community in Salt Lake was amazing just knowing that you were in the same boat with all these developers. Either just getting into Solana or being in it just recently. Learning Rust and learning the runtime environment and what is possible on Solana is really crazy. And everyone was trying to help each other and answered questions. And if you didn't know the answer, they would direct you to somebody else. And everyone's just like, "yeah, let me help you with this," which is my favorite part about that.

Nick (16:48):

Yeah, it's been really cool to see communities pop up. Also happening right now is like AthensDAO in Greece. And unfortunately we weren't able to make it there, but I think we'll see, over the coming months, a few more of these communities start to pop up that are a bit more like longer running than just the kind of week long hacker house format.

Chase (17:06):

Yeah. I'm a big fan of the community run hacker houses and all these sorts of things like mtnDAO, just because whenever it's built out with the community like that, it just forms this other type of bond with everybody and it's just really exciting to see all that happen.

Chase (17:23):

This is the point in the show where we shift gears a little bit. We talked about that excitement, how Cronos came alive, you guys winning Riptide and now I want to talk about what that experience was like for you guys. Because this is the very important part of this show where we talk about what sucked and what was good and what could be better. So, I want to start with Elias this time. You came from a front-end engineering background? What's actually the languages that you had touched before you came to start building on Solana?

Elias (17:58):

Even before I was a front-end developer, I was dabbling in data science for a bit. It was a lot of fun, but a bit too meticulous for my taste. So I was dealing with a lot of Python. Fast forward to when I graduate, I was really interested in front-end development, got pretty good at helping with some friends and building toy applications and TypeScript React, some toy web apps with Next.js. And then that's whenever, like I said, met Nick, joined the team and I was building front-end applications for a while at, like, six months. And then I found, like a lot of people who started their Web 3 journey, Nader Dabit's Ethereum article on how to... It was like a super simple... I forget exactly the context of what the project was, but it was on dev.to. And read through it and tried to understand what is this environment? What is this dev environment? What's going on? Not too long after I found Solana and Nick also brought it up, was like, "Hey, we should maybe look into this." And then-

Chase (18:53):

Did you do Dabit's tutorial on Solana too?

Elias (18:55):

I did. Yeah, I did. Yeah.

Chase (18:56):

Yeah.

Elias (18:57):

It was a lot simpler. I don't know. But also more difficult in some ways. When we were working on Factor, Nick gave me the talk like, "Hey, we may not need front-end developers. So there's a chance that I need you to flex over to becoming a Rust engineer, which is, you know-

Chase (19:13):

So he didn't fire you?

Elias (19:15):

No, no he did not. Luckily. Yeah. So fast forward to like mtnDAO when we know finally realize Cronos has a lot of potential. I buy the book that a lot of people seem to have and I have it on my desk right here, the programming Rust book. And it's been my north star, I would say, as far as growing my skills as a Rust engineer, as well as living in the Solana repo and Anchor repos.

Chase (19:39):

So you guys are building this in straight Rust? Are you guys also using Anchor?

Elias (19:44):

Yeah. So in the core of Cronos, it's a lot of Anchor. A lot of what I deal with, I'm building and optimizing the Geyser plugin that we have to listen to Cronos accounts and execute tasks when needed. That's just built in Rust and other like asynchronous libraries and things like that, but not Anchor specifically.

Chase (20:03):

This is the part, the glass chewing, what was your learning curve during that process of learning Rust coming from front-end? Was it as painful as everybody says? Everybody's different on this front, so what was that like to learn Rust?

Elias (20:16):

A big mistake that I would advise people attempting to get into the space would be first of all, just learn Rust by itself first. At least start there and understand that it is different from Anchor. And it's just a framework that lives with Rust. And then try to understand the Solana runtime just a little bit. And those are three separate entities, but they all coexist and you need the three in order to make a simple to do app in Rust on-chain. So differentiating between those three different entities is really important. And if you just jump straight into a Solana Anchor project, not knowing Rush, you're going to get really confused and pretty frustrated.

Chase (20:52):

So for you, was it hard or was it just time consuming? You just had to grind it out and you learned along the way?

Elias (20:58):

Yeah. Yeah. It's one of those things that you just have to do every day. You have to, you know, for me every morning before we would go to the mtnDAO-

Chase (21:06):

Glass for breakfast.

Elias (21:07):

Yeah, I would literally just sit on the couch and wait for Nick to finish showering before we drove to the office. And I would just read a chapter of the programming Rush book and it would just go over super simple things like structs and basic functions. And if you're a software engineer, it's not too difficult to transition into Rust, it's just another programming language just in the different context. And it looks a little weird with two semicolons next to each other or whatever, syntax. But it's not too bad. One of those things you just got to do every day. And then before you know it, you'll just hit the road running. It's pretty nice.

Chase (21:39):

I think a lot of people talk about chewing glass, like it's actually Solana, that's the real glass chewing, about learning the native concepts, like using PDAs and these things. And there are people out there who just don't ever end up learning Rust and they never actually tried it. They never just sat down and did it. You could get a little bit resistant to it just because it looks so foreign.

Chase (22:00):

And then the other part is, I've done a couple of Twitter Spaces around this exact same thing about you saying start with Rust. That's my recommendation always start with the base layer before you're using any sort of framework or anything that's intertwined in it, but there are people out there on the other side of the camp that say, just start with Anchor. I obviously disagree, because I think learning that first base language is always going to be the best. And it's going to save you down the road when you're running into issues.

Chase (22:27):

I'm going to go ahead and ask you Nick, kind of the same question, what your experience was like? What did you do to learn it? Was it similar to Elias? Was it hard? Was it easy? Was it just time consuming? What would that look like?

Nick (22:39):

I had some background working in back-end systems. Yeah, for my time out in California, I had worked mostly with Go prior and actually first tried picking up Rust in 2020, because I'd seen it was the most popular language on GitHub and it was just like, what is this? And I actually hated it the first time I looked at it because I was coming from that Go world. And Go is designed to be super ergonomic and easy to read and talk about and communicate and Rust is more optimized for performance,

Chase (23:13):

Performance and pain.

Nick (23:16):

Yeah. And so I hated Rust when I first looked at it, and I pushed it off to the side and didn't actually look at it again until we dove into Solana. I've since come to love it. It is a little bit steeper of a learning curve and there are some extra pieces to the mental model that you need, in terms of understanding memory and ownership of variables and how all that stuff works, lifetimes, for example, that other languages don't have. So that makes it a little more complicated or harder to learn. But it's not anything that can't be overcome, I think. It's just another programming language.

Nick (23:50):

But yeah, definitely breaking apart, as Elias said, the difference between Rust problems, Anchor problems and Solana problems and understanding that these are all like three different systems is probably the hardest thing when you're first diving into Solana because it all looks the same and all the error messages are cryptic and if you don't have a whole lot of debugging experience, it can be hard to pull that thread because all this stuff is quite new and a lot of devs, I think have the pattern of, you get an error message you don't understand, copy it in a Google and see what Stack Overflow results come up. Usually we're running into problems that no other devs have run into yet. And it's just like-

Chase (24:28):

It's actually pretty cool though. Like to be one of the first group of people on the planet. You guys are going to be the ones who answer these Stack Overflow questions in the future because that always starts somewhere. The first guy had to just figure it out.

Elias (24:40):

It's cool, but you're like, "I don't know what to do now." I guess we're just going to have to figure it out. So that's where I would just go to the Solana codebase and Nick has recommended multiple times. Just go live in there, you'll understand the runtime environment better, your errors will be easier to debug. It's a lot. The Solana codebase is a lot, but there are parts of it that really help you understand what is going on underneath.

Chase (25:03):

A lot of people come from Web 2, and again, I'm one of those people. We're used to having our hands held. We're used to being able to find the answers we want. We're used to all these pretty, amazing tutorials and all these different things. And when that's not the case, it makes it a lot harder. Sadly enough, not everybody's this reverse engineering code diver that's going to go do that sort of thing. And it takes a lot of time and a lot of effort and sometimes, at the end of the day, there might not be considered the greatest payoff for all that work. But the true engineers, the ones who just like to figure shit out, are going to go do that and then they're going to figure it out and then they're going to build Cronos. So it's awesome.

Nick (25:43):

It's definitely how you know you're on the bleeding edge, is when Google doesn't come up with any results for your error message.

Chase (25:49):

Zero results.

Nick (25:51):

Yeah.

Elias (25:52):

The beautiful, no search results for that Google search and like, well, okay cool. Like whatever.

Nick (25:57):

Yeah. I Would say to any devs that find themselves in this situation, the Anchor discord in particular is my new Google for trying to find solutions to these problems. And usually, 70% of the time, someone has asked about some of the error problems like we're running into in the Anchor discord somewhere. And there's been someone that is able to chime in and help and Armani and the team that's there is extremely helpful in terms of answering questions and generous with their time.

Chase (26:27):

And Alan and everybody that's out there and Jacob who's on our DevRel team and Donny. There's so many people and these are guys that are actually working on like Serum and Anchor and Solana Labs and all that stuff. But outside of that, the community of people just helping each other solve these problems, it's amazing to watch it happen in real life.

Chase (26:48):

We've been going on for a while now and I want both of you, if you can, to tell me what is missing from Solana right now, like in tooling? You guys just created one that was missing, where you were seeing a lot of tooling come out.

Elias (26:59):

I have one.

Chase (27:00):

Okay. All right, we'll start with you Elias. What are we missing right now?

Elias (27:04):

I'm probably stealing this from Matt because he's probably thinking about it, but one thing we've ran into recently is DevOps pipelining. It's pretty difficult to handle versioning from so many different projects. And when we're developing, we're having to stay ahead of Mainnet and work on Testnet and it's a very complicated, and different projects do it differently, but right now in our repo, we have a forked version of Anchor just so that we have up-to-date versions of Anchor, but using some different-

Nick (27:32):

The latest Solana dependency versions.

Elias (27:34):

Yes. So that right now is something that we have to build, but if there's a way to do that at scale for a lot of other teams, that'd be great.

Chase (27:41):

Yeah, that's not the first time I've heard that one, but that's a good one. I hear it just because I'm always paying attention to a lot of different places, but I don't know if everybody, except the ones who are coming into this problem actually know that this is something that's kind of necessary. It's not one of the ones that people are most vocal about. It's usually error codes and indexers and all these things. So Nick, did Elias steal yours or do you got something else for us?

Nick (28:05):

No, I think that's a great one. There's, at least for what we're doing, where we have both on-chain programs and a plugin, that we're trying to ship DevOps challenges around keeping the versions in sync between those two pieces can be challenging. And then, I guess something that's been on my mind a little bit is how there was a DeGit project in the Riptide Hackathon, decentralized Git, which I think, stuff in that space like decentralized DevOps processes and how does a decentralized global team of engineers contribute to a protocol and how do you keep the community open, but also secure, is, I think, an unsolved problem at this point.

Chase (28:50):

Well, I look forward to the Cronos team actually building out this suite of tools, all of it.

Nick (28:56):

A few pieces, but yeah.

Chase (28:58):

Yeah. So I usually wrap these shows up just asking what advice would you give to somebody who's thinking, on the other side, "Hmm, maybe I'm about to jump into Solana. I'm not sure if I want to put in the effort to build something." What general advice would you give somebody who was going to build or is building on Solana right now?

Elias (29:17):

If you're frustrated with learning Rust, but you're really wanting to build on Solana, then you're doing it right. You're not doing it right if you're not frustrated, that's the chewing glass part.

Chase (29:27):

Yeah.

Elias (29:27):

Just keep going. Because at some point you'll be able to look at other projects and their smart contracts and go, "oh, I see what they're doing." Like right now I'm looking at the Holaplex, that's called RabbitMQ plugin or Geyser plugin, shout out to the Holaplex team, and trying to understand why they made certain engineering designs with their plugin and see what we can take from. And that's just the beauty of open source of course. But yeah, if I wasn't chewing glass consistently and I wasn't looking at code and other repos, then I wouldn't be able to do that.

Chase (29:58):

Yeah. That's awesome. And like it is, I wish everybody would start open sourcing their code out there, but we'll get there eventually. How about you Nick, what kind of advice do you have? And again, we've kind of talked about a few good ideas for the community, so what do you think?

Nick (30:13):

Yeah, I think probably two things, as Elias mentioned, spending time in the Solana repo, it's helped a lot. There's a lot of patterns in there that, if you're trying to get familiar with Rust, it's a great resource to learn from. And the second thing is to actually read the error messages that you get back, because when you actually pull on that thread, they are very cryptic error messages a lot of times, but they do have information that leads you to the bug and the problem or points you in the right direction, maybe is the best way to put it. I find that skill, that debugging skill is like a muscle that needs to be trained and learned and doesn't always come supernaturally, because it's just hard. But yeah, reading error messages and trying to decipher what they're telling you is a fundamental exercise to dealing with large complex systems.

Chase (31:10):

Yeah, and it's also just a really cool skillset to have, to be able to just do these manual debugging stuff. Yeah. Like you said it becomes like a natural kind of mental muscle that all of a sudden, now it just happens quite naturally, once you get to a certain point.

Elias (31:23):

Yeah. One thing for those interested in just in general, distributed systems, trying to understand Solana a little bit better from a higher level, there's a great YouTube course from MIT. If you just search "distributed systems MIT", it's an OpenCourseWare, like 12 lecture series, just to understand like RPCs, multi-threading, concurrency, consensus and things like that. It's really beneficial to understanding distributed systems, blockchains, well, not necessarily blockchains, but at least for Solana distributed systems.

Chase (31:55):

Awesome. Yeah. There's a lot of people that come into blockchain and they don't even really know what a distributed system is. And then a lot of the times it's like, hey, go actually read about like what this thing is before diving into this.

Chase (32:07):

All right guys. Well really, really, thanks for coming on the show. I'm glad that we got to catch up. Congratulations winning Riptide. I'll talk to you later.

Nick (32:16):

Yeah. Well see you in Austin.

Elias (32:17):

Sounds good. Yeah, see you in Austin.

Chase (32:19):

All right. Cheers.