Ep 5: Data mapping and Simple Wikipedia

0:09Okay. Hello. Welcome to episode four of

0:10Curiosity Build series. My name is

0:11Trevan Hauck and I am going to learn in

0:14public and fail before your very eyes

0:16while I attempt to use AI tools through

0:19the amazing

0:22program of Cursor, Chat, GPT, Claude,

0:26Grock, etc. and try to create something

0:29with my curiosity. In episode four,

0:32we're going to bring up the Borderless

0:33Wire website that I've been working on.

0:35I've already got the localhost ready to

0:38I've already got the localhost ready to

0:38go. So, let's take a look what this

0:40website is. BorderlessWire. It is my

0:42website is. BorderlessWire. It is my

0:42vision for a news and fact website uh

0:45that spans the globe. So, what we've

0:47done is we've got a globe here. The last

0:49episode we added color and labels and

0:52worked on the sidecards. The goal here

0:55is you should be able to click on a

0:56country, get some quick facts probably

0:58from Wikipedia. And then the cool part,

1:02the part that's like got me interested

1:04the most is news features. So I want to

1:06be able to click on a country, you know,

1:08like Algeria, and get some news sources

1:12from that country so I can look up and

1:14read stories that are being written

1:16about by the people living through them

1:18or around them and have that translated

1:21into my own language. There's a lot of

1:23hurdles there with copyright and

1:25sourcing that I need to figure out. But

1:27first, we're going to work on the

1:29interface. So, I'm not going to do too

1:31much today. I want to work through a a

1:33poll request to GitHub and kind of show

1:35what that looks like. Uh, but first, we

1:38do want to make a little change on here

1:41because I don't want to just not do

1:42anything. So, first, I think getting a

1:46better quickfax profile would be nicer.

1:50I'm not sure how much I'm allowed to

1:52bring over from Wikipedia. But for now,

1:54we'll just go through the motions to

1:56have it displayed. So, I'm going to go

1:59back over to my agent here and we're

2:01going to talk a little bit about but

2:02we're going to put in planning mode

2:03first because it's better for it to plan

2:06so it can tell me what it's doing and I

2:09can make suggestions and the agents can

2:12ask me questions.

2:14I want to pull in more facts from

2:16Wikipedia under the quickfax site. Is it

2:18possible that we're able to bring over

2:22the side profile for a country from

2:25Wikipedia that lists those quick

2:28population size and government size,

2:31whatever that Wikipedia lists next to

2:35the main articles on its site. The

2:38quickfax that are loading now is great,

2:40but if we could also include that, I

2:42think that would be a richer experience.

2:48So, let's see if it's able to do that

2:48for us. I am also going to bring up the

2:50preview here so we can watch it in real

2:53time. Um, oh, it's going to bring up

2:56this first. Let me grab the URL

3:05and just open browser here. Do it that

3:05way. We'll do it that way. We'll do it

3:08live. All right. So, it's planning. It's

3:11asking me some questions. Which

3:13Wikipedia side profile fields should we

3:15include? Pick all that matter. Well, you

3:17know what? Why don't we look at

3:18Wikipedia?

3:21We'll just look at Wikipedia Canada

3:23since I was just there. I do like, you

3:27know, this stuff.

3:31Interesting.

3:34So, I will instead of picking all these,

3:36I guess, let's see.

3:39Capital government type

3:43I think is good population

3:47time zone

3:50I also think what else would we want to

3:52know I think GDP

3:56um

3:58would be nice

4:00let's just do GB GDP uh PPP

4:04purchasing power parody is what that

4:06stands for everyone uh

4:10currency would be kind of cool.

4:13Where should this show extension side

4:15panel both web and extension? We'll just

4:16do web app only. Is it okay to parse

4:19Wikipedia info box html best or prefer

4:22wiki data more structured not identical

4:24to the wiki laid out? Uh

4:38let's

4:38do this so we're not trying to reinvent

4:40the wheel and see what that looks like.

4:43Uh yeah, we can always change the

4:45features later what we want to bring in.

4:47It'd be cool if we had like the the

4:50flag.

4:59Is it possible to also pull up the flag

4:59image as well when we show this? and

5:01potentially the motto

5:04only because I saw it right here while I

5:06was talking. That happens to me

5:08sometimes like I'll I'll be saying

5:10something while I'm looking at something

5:11on the website that I'm working on and

5:13then all of a sudden I'm like, "Oh yeah,

5:15that too." Right. Should the flag model

5:17come from the same Wikipedia infoscrape

5:19or should be shown on QuickFax panel on

5:20the web? Yes.

5:22All right. Now it's going to plan. And

5:25while it's planning, I'm going to we'll

5:28usually sit back and let it work, but

5:30it's going pretty quick here. So, it

5:32lists the plan here for us to read, the

5:35scope, the key files. It's going to

5:38interact with the approach, some notes,

5:42tools, and to-dos, and then it lists the

5:44to-do list in the order. Um, we can

5:46always add another to-do if we want to.

5:48So, it's going to fetch the data from

5:49Wikipedia, store it in Supabase in the

5:52local cache, and then render everything

5:53on there. Um, I do want to add

6:02do a visual check at the end to make

6:02sure everything renders correctly and

6:04there's no overlap. We want it to look

6:06clean. If that entails adjusting the

6:09size of the display box, you're okay to

6:12do so.

6:15All right. So, we added another one and

6:18we'll click build. We'll see what this

6:20baby does for us. And since we're doing

6:22that, we will go back to our localhost.

6:26that, we will go back to our localhost.

6:26And we want to see the changes happen.

6:28So why don't we click on Canada

6:30and see what it brings up. And I noticed

6:32I have to do this to get the whole thing

6:35to show up here. So we shall see. It is

6:39building. I will probably

6:43speed this up so we don't have to sit

6:44and wait.

6:46So we'll be back when we get close to

6:48finished. Okay. So, it's working through

6:51its planning. It's a lot to go through,

6:55but it's done two of its to-do lists,

6:59um, two of its to-do items, and it's

7:01working on the last couple, including

7:04the preview that I gave it.

7:07Now, let's see. If the Supabase table

7:11doesn't yet have a new inbox columns,

7:13the upsert will log an error, but the UI

7:15will still show live info box. Next

7:17steps, run the web app. verifying

7:18country clear box. If you want me to

7:20back fill or migrate Supabase table

7:22scheme, I can prep that. So, let's see.

7:24Is this I don't know if this did

7:25anything that we asked it to do.

7:50Canada.

7:50Just to double check, we'll do a

7:52refresh.

7:54Nothing coming up, it seems like to me.

7:58Yeah.

8:03So, I went to refresh the page and I

8:03open up the

8:05quickfax. Actually, you know what? Just

8:07to double check ourselves, let's go look

8:09at the actual website with Canada and

8:13see what uh what comes up here.

8:17Canada.

8:19Okay. So,

8:22pretty much nothing.

8:24The quality is going to look worse on

8:26the local preview. Don't forget that.

8:27So, don't be alarmed. Telling that to

8:29myself. Uh, okay.

8:32I refreshed the page and opened it up

8:34and nothing looks like it's been

8:35implemented. It looks like you might

8:37have updated files, but I'm not seeing

8:39it displayed. I just have the same exact

8:42quick facts when I am looking at Canada.

8:47I'm not going to look through every

8:48country, but I'm assuming it's all the

8:50same. So, go back through and check why

8:53this isn't showing up.

8:56It could be something to do with Super

8:58Base, but I am getting over my skis

9:01there.

9:07All right, let's see what our agent

9:07thinks of that. And remember, we can

9:09always switch to a different agent if

9:11you find that like I I find that if I

9:13get stuck with Codex, I'm going to

9:15Claude. Um, I try to reserve Claude

9:20because it does things really well, but

9:23it uses a lot of tokens and if it makes

9:25if it makes an assumption and starts to

9:27do something, it'll spend a lot of

9:29tokens wasting time. So, it's better to

9:31work through something else that doesn't

9:32have that kind of draw down on my usage.

9:36So, I try to reserve it. I tried to

9:39reserve it.

9:44I don't get all these other things

9:44popping in there.

9:46Okay. Hard refresh.

9:50A hard refresh. Huh.

9:53Well, I'm not seeing any hard refresh

9:56available.

9:58Let's see. Maybe you can't hard refresh

10:00here.

10:03Okay. All right. All right. All right.

10:05All right. All right.

10:07Okay.

10:09The problem is all of this information.

10:11It's too much coming in, but that's not

10:14bad. Let's look here.

10:16This is the regular website. So, regular

10:19website, Canada,

10:22new website or update we're trying to

10:25build. Let's do a refresh here.

10:33Canada.

10:33Okay. Okay. Okay. Okay. Okay. Really

10:35like this. We just need to get rid of

10:37whatever is going on here. So, let me

10:41copy that.

10:44This looks great. The only problem is

10:47when we look at the capital section,

10:49instead of just saying the capital city,

10:51there's a bunch of other codes or

10:55inline image text

10:58as well as uh latitude and longitude. We

11:03don't want any of that. We just want it

11:05to say the capital.

11:11Okay.

11:11I like that.

11:20Still need to work through the, you

11:20know, fair use of this, but that's fine.

11:24Um, this isn't public

11:26anywhere. It is here on my machine. It

11:29is not pushed to the website yet. Um,

11:34right. You click on Canada just shows

11:36that need to learn a little bit about

11:39what's allowed. Okay, let's try this

11:41again. Canada quick facts,

11:45please. Refresh. Oh, refresh.

11:49Refresher.

11:50Sorry. Canada.

11:53Quick facts. Still doing that. Remember,

11:56we can always pop the issue open, copy

11:59it, and paste it.

12:02In addition, because that might not be

12:05what the only problem here

12:09got that issue. In addition, I still see

12:13Ottawa, but then MW-par-output,

12:18etc., etc. A bunch of extra things we

12:20don't need.

12:23All right, but look at this. It's

12:26getting there, right? It's getting

12:27there. I like it. I like it. I like it.

12:29I like it.

12:32Big site here.

12:40Getting this functionality set up, how I

12:40want it to be displayed and delivered.

12:43The next piece is the fair usage part

12:48and understanding that and then getting

12:50news sites stood up.

12:54And then after that, I think we'll do

12:56the translation part. And then finally,

12:58a better wrapper for the whole thing so

13:00it looks cooler, right? I'm trying to

13:04get these things to be more legible, to

13:06be more interactive, to give information

13:08I want and to look nicer, but this isn't

13:11going to be the final UI, right? We're

13:14just this is our big sandbox basically

13:17that we're playing in. So, let's see if

13:20it even accomplished that. We're still

13:23stuck

13:25on this extra stuff. I wonder what all

13:27that stuff is.

13:29Got rid of some of it.

13:32Let me show a picture because it can see

13:35photos. Um, all right. Take a picture

13:38there.

13:44Paste it in here. Oops. See, I did not

13:44copy the picture. I need to grab it.

13:47Give me a second here

13:50to grab that file.

13:56Okay,

13:56screenshot. Drag it in here.

14:04That's a screenshot I just took of our

14:04site

14:06local preview. You can see there's so

14:08much extra text after the capital of

14:11Ottawa and it's just not making sense

14:14why that would be there. I'm not sure

14:16why that's showing up. It's a mistake on

14:19our part somehow.

14:22We need to remove that and make sure

14:25it's applied across every country.

14:38Okie dokie. Artichoke.

14:38Once we get that sorted, I am going to

14:40push a PR

14:42um

14:44a poll request which basically puts this

14:47up to get checked to make sure it's good

14:50to uh make live and then uh yeah, won't

14:57completely publish it, but we'll get

14:58we'll get close through the motions.

15:01Those issues are coming from inline

15:03content embedded inside the capital

15:05cell. I've updated parser to strip it.

15:09Okay. So, it tells us where it put that

15:11in there. And hopefully it'll do that

15:14every time now. Canada. Give me Ottawa.

15:17Oh, bingo.

15:20This is what I want. I want capital

15:23currency facts.

15:25Okay. US. Oh, we got to refresh.

15:29How about Algeria again?

15:33Okay. Love it. Capital government time

15:36zone currency. Great.

15:41China

15:47doesn't have it.

15:47Why not?

15:54Why not? Why is it just there? Is it on

15:54every country or just some?

15:57It's a lot of work to do every country

16:00under a dictatorship.

16:02Okay. Greenland.

16:06Why? Why was China holding up over here?

16:09Just a late update.

16:12Cambodia or Vietnam?

16:17Is it only China? I'm not going to check

16:17all 200 and whatever countries. All

16:20right, let's go ask it.

16:24Double check in here, too.

16:31So, I've checked about six or seven

16:32countries, and they all look good,

16:33except China. is not looking like it's

16:36updated. How can we be sure that we've

16:38gone through and applied this update to

16:41every country

16:46and I'm going to ask it to plan that

16:46because I want to know how it's going to

16:47do that. I don't want it to just do it.

16:50It's too hard to check everything and

16:51just I don't want to assume

16:54client side refresh backfill countries

16:57and Supabase with onetime script both

17:00client refresh and superfase backfill.

17:02We'll do that. I like that.

17:13So, refresh add onetime backfill.

17:13Yeah. Yeah. Yeah.

17:16Yeah. I like keeping an eye on confirm

17:18missing info box. Refresh logic. One

17:19time supervis backfill document. How to

17:21run a backfill script. Yes, indeed. All

17:23right. Off to the races with you. And

17:26we'll go back to our browser here while

17:29it builds. Should be a quick build and

17:30then we should be able to commit and and

17:32push this out.

17:35One would hope. Okay, I'm back. It

17:38hasn't done much but thinking and

17:40planning for a few minutes. It's taking

17:42longer than I anticipated, but that's

17:44fine. It's being thorough, but it should

17:46be close to executing the last two steps

17:49here. And once we've got confirmation on

17:52that that China's got all the

17:54information it has and we've done our

17:57back fill, we will go ahead and commit

18:00and do a PR as promised.

18:11All right. So much going on. So much

18:11going on. But I love it. I love how this

18:13is looking. I like that we get the

18:16profile here pulled up. Hopefully, we'll

18:18be allowed to keep that. And then the

18:21next thing would be the news.

18:23This is going to be a lot of uh of work

18:26for me. Who knows how long that will

18:28take and what is allowed and what's not

18:30allowed, but I'm excited. I'm excited to

18:32explore and learn and try. So, it says

18:36it's done it. Let's look at China.

18:39I'm not seeing it here.

18:50We didn't do a hard refresh, didn't we?

18:50Oops, wrong country. But

18:57I wonder if there's something going on

18:57with China Wikipedia specifically.

19:00It's not any other country it's having a

19:02problem, is it?

19:04Haven't seen a single other issue. No, I

19:06bet it's just a China thing.

19:09Interesting.

19:26Well, why don't we look at uh Wikipedia

19:26on China right now? Oh, maybe they don't

19:28have Oh, because it's simple Wikipedia

19:31and simple Wikipedia might not have

19:34what we need. And simple is good. We I

19:36we wanted it to pull from simple because

19:38it's easier to understand. So I think if

19:43we go to regular regular Wikipedia

19:46main page, type in China and see.

19:51Yeah,

19:54I guess they don't have a

19:56profile over here. I wonder why.

19:59Oh no, it's from Simple Simple. Oh, the

20:01People's Republic of China. Maybe that's

20:04why it's not searching up the actual

20:07country name. That's a good thing to

20:11realize

20:13um

20:15because this is still in simple right

20:16okay so we need to make some

20:18clarifications

20:20I just realized that when you linked to

20:24China it's going to China

20:27as

20:28the region from all time but the country

20:34information and the quick facts like the

20:38government and the capital and the flag

20:41are coming from the actual official

20:43country page. And for China, that page

20:45is called the People's Republic of

20:47China. So, we need to be careful that we

20:50don't have this problem in other

20:52countries. We want to make sure we're

20:53using the country page that's attached

20:58to the government and the country facts,

21:02not

21:05a page that is more about the

21:10culture region or the ancient

21:12civilization of a place.

21:19Yeah, I didn't consider that. Um, but

21:19that makes total sense that China would

21:21have a Wikipedia page, but then the

21:22People's Republic of China would because

21:25China is going to cover all of the

21:26different dynasties and iterations of

21:30government and

21:32time periods throughout the history of

21:33the region. But the People's Republic of

21:35China is going to be a page just about

21:37the current government, and that's

21:39usually attached to the flag. How should

21:41we choose the Wikipedia page when the

21:43country name is ambiguous, like China?

21:45Maintain manual mapping list for known

21:47ambiguous countries. Use wiki data to

21:49resolve the country entity and then

21:50fetch the Wikipedia page. Try the

21:52country main then fall back to

21:54disambiguation.

21:56I think

22:03I think we'll just use that. Should this

22:03be applied to both summary info box or

22:05only the info box?

22:08Yeah, both the summary and info box use

22:10the same page. I like it. So, this might

22:12take some time, but this is a good catch

22:14and this is a good learning moment for

22:17me.

22:18Um, yeah, in case you're curious, simple

22:22Wikipedia is just regular Wikipedia that

22:25uh is simplified, easier to understand.

22:28Um, if you go to the main page of

22:29Wikipedia. Oh, we are here. Where's the

22:32little like Yeah, here we go. So, in

22:35simple English or no, we don't want the

22:38main page of Wikipedia. We want to go to

22:40Wikipedia regular

22:43the Wikipedia foundation. And then we

22:46want to go to Oh, no. This is donate.

22:48Oh, it's give to Where's your regular

22:53Where's your regular page? Maybe we just

22:56go to Wikipedia without simple in front

22:57of it. Probably the fastest way for me

23:01right now.

23:03So, Wikipedia classic, right? But in

23:08Wikipedia,

23:09there's so many different languages that

23:12you can um access, right? But they also

23:17allow you to access simple Wikipedia.

23:20And there is a page that shows the

23:22different articles. Yeah. So here

23:23they've got a million plus articles in

23:25these languages. 250,000 plus articles

23:28in these languages. So simple English is

23:31one of these. So, while English is like

23:33the founding uh language for Wikipedia

23:36and English Wikipedia has got millions

23:38of articles, simple English doesn't have

23:40as many, but it's very helpful,

23:42especially if you're learning English or

23:44you're working with a student or you're

23:46just trying to understand something

23:47that's really complex. If there's a

23:49simple Wikipedia page on it, that's

23:51where I would go to get my framing.

23:54Anyway, that's where we get this

23:56information from. So, it is working and

24:00it's definitely going to be a bit of a a

24:02work for it over here. So, I'm going to

24:05probably dip out for a second and come

24:07back when we get a little more along

24:10with our fix here.

24:40Okay,

24:40we are hopefully

24:43where we want to be. So, let's take a

24:46look. Let's refresh the page here

24:49and let's click on China.

24:58Hey. Hey.

24:58Good job. Good job.

25:02We'll just go and check this.

25:05We'll click China.

25:12Hey. Hey. All right. So, this is the

25:12local preview. This is what the actual

25:14site looks like right now. The live

25:16site. Look at the difference. Look at

25:19the difference.

25:21Look at this difference. So, I'm very

25:24happy with the change. So, this is

25:26currently what it looked like before and

25:29what's on there now. And this is the

25:30updated version.

25:33Little more information, little little

25:35uh little nicer, little familiar, little

25:36Wikipedia profile. So, we are happy with

25:40that. What we're going to do now is

25:42we're going to commit these changes.

25:44Let's commit these changes and prepare

25:46for a PR.

25:49Actually, I'm going to do a PR with

25:51Claude. So, we'll just do let's commit

25:53these changes. And then I'm going to

25:55open up a new server down here and type

25:57in Claude because we have a PR skill

26:00that uh I actually am not sure if it's

26:03something that just existed and then we

26:04imported or if it's something Daniel put

26:06together. It's probably maybe a little

26:07of both. Um yes, include them. And it

26:12does a really good job sending a poll

26:14request in, doing reviews, um tagging

26:17the people that need to be tagged. And

26:19so we're going to use that skill in

26:21Claude. So it's committed. Want me to

26:24push this branch? Um, no. So I want to

26:30create a pull request for the commits

26:33that I just made for BorderlessWire.

26:37that I just made for BorderlessWire.

26:37And so now Cloud's going to open up its

26:39skill package and do this. It's

26:42gathering the current state of the

26:43branch, looking at everything that we

26:45did, and then it's going to run the

26:48skill to push this up to GitHub, which I

26:52already have connected and linked to my

26:54account. So, it can do that. You

26:55wouldn't be able to do this without

26:56either having access through GitHub or

26:59or there's like a if the branch is

27:02public if you're working on something

27:03that's public and open, but this is

27:04private. Um, and this is connected just

27:09to my GitHub.

27:12Once it's there, pull request goes

27:14through and does checks uh to make sure

27:16everything's working correctly. I

27:18believe they also have Copilot go

27:19through and do a check and then it's

27:21connected to our Verscell account, which

27:23is where we host our websites where this

27:25is hosted at. And it's built over there

27:28as a preview. If it looks good and I

27:32like everything, I can push it to

27:33production and then it'll be on the live

27:35site in about 20 minutes. All right, so

27:38we're going to check this out. um not

27:40check it out like look at it. It's

27:41called get checkout which is basically

27:43like taking everything that you've put

27:45together in your shopping cart and going

27:47through to uh where you want to purchase

27:50it and get out.

27:53Clunky metaphor, but

27:57yeah. Yeah.

28:08It's going to check out the main and

28:08then

28:11created a PR. So, I can click this and

28:14it'll take me to the poll request,

28:17right? Enrich country facts, Wikipedia

28:19info box, all the things that I want to

28:21do with it. And uh yeah, yeah, cool

28:26stuff. So, stay curious, don't forget to

28:28commit. And we'll be back with another

28:31episode doing more features soon. Have a

28:33swell day.