Using a chatbot to "deep research" genealogy

[John Winn]

I fear that in a very short time this kind of "tool" will just be accepted by an ignorant, lazy society as providing a quick, truthful, answer to many things. As most of the masses already only 'know' what they see on YouTube how would they know they don't really know if they don't really know anything to start with ?

Won't be long, methinks, before it will get very difficult to verify many things, certainly current events. We're going to become true sheep. Genealogy is the least of our worries.

Sorry - bad John made me have another glass of wine and make this post.

 

[CWdiaryPA]

I fear that in a very short time this kind of "tool" will just be accepted by an ignorant, lazy society as providing a quick, truthful, answer to many things. As most of the masses already only 'know' what they see on YouTube how would they know they don't really know if they don't really know anything to start with ?

Won't be long, methinks, before it will get very difficult to verify many things, certainly current events. We're going to become true sheep. Genealogy is the least of our worries.

Sorry - bad John made me have another glass of wine and make this post.

We're all already true sheep, just a matter of degree, imo. Brings to mind back when a train was seen on the screen in movie theaters for the first time. Everybody jumped back so the true, real train wouldn't run them over. They ran, stampeded out the door to safety.

Or did they?

Lumiére brothers, 1896. Just looked it up but can't get a clear picture of the event. What do we ultimately know about anything, after all? This post brought to you while under the influence of Tofu Dip.

 

[13 IL Cav & 64 GA Inf]

I came across this article yesterday that left me with less information than I had when I started, which is a nifty trick for a website that says it's there to 'guide' readers. The piece describes how author decided to use ChatGPT, an open source chatbot, to research her family tree. Her stated goal was to do "deep research," which is the term ChatGPT uses for it's search and analysis tool. Supposedly this tool will take your prompts, search the web, analyze the material it gathers, and provide you with a synthesized report that gives you results without doing the work.

The author then walks us through her prompts to the bot, which, no lie, start with "Provide information on the Smith family from Lancaster, Pennsylvania, between 1800 and 1900." This is her specific example after she helpfully points out that "Tell me about the Smith family." is not specific enough. Ya think?

15 minutes later, we are told, the author has her report. But she never tells us anything substantial about her results. Almost as a throwaway she writes "I really enjoyed knowing that my ancestors came over on the Mayflower." Huh? Setting aside the grammar (Does she now no longer enjoy knowing that? Or has she stopped knowing it?), the leap from 'Smith family in Lancaster' to "came over on the Mayflower" is a heck of a jump. Now, honestly, the Mayflower passenger family lines are incredibly well documented so I suppose if you were descended from one of those families it would be easier to find information on your genealogy than if your ancestors were potato famine era Irish immigrants. But anybody with a Smith line in their family knows you have many, many ways to get bad info in your research.

The article includes a number of caveats. I loved this one - "While AI offers powerful tools for genealogical research, it's important to be aware of its limitations, for instance, accuracy concerns. AI-generated content can sometimes include fabricated information. Always cross-reference AI findings with primary sources or established databases." She ends with "Be sure to critically evaluate the AI-generated information and ask more questions. As with all research tools, combining AI capabilities with traditional methods and personal insights will yield the most comprehensive and accurate results."

In other words, use our tool, get a report which may be bogus, and then do the work of checking existing sources and research. Okay then.

I honestly think that tools like this, especially if they can scan the deep web, do have some value in them. An experienced researcher could use a tool like this to look and see if there are sources he or she might have overlooked. That researcher would use the results as a prompt to check new sources. But, as always happens with tools that promise short cuts, folks who aren't experienced are going to put in some simple prompts, get back an unreliable report, and believe it. I can only imagine how much of this junk will end up on Ancestry and Find a Grave and then get copied onto multiple trees.

<sigh>

I just tried ChatGPT deep research to dive into my family history — here’s what happened

AI and me

www.tomsguide.com

IF I had access to an AI tool that I trusted not to be hacking me, I would ask it a question about several generations of a family line for which I already KNEW the answers, to check the AI tool's ability to get things right. IF such a TRUSTED AI tool got the questioned generations and their facts right, then I would try using it to discover facts on some partially known and related unknown generations. (I would still be very wary of the results.)

 

[13 IL Cav & 64 GA Inf]

I fear that in a very short time this kind of "tool" will just be accepted by an ignorant, lazy society as providing a quick, truthful, answer to many things. As most of the masses already only 'know' what they see on YouTube how would they know they don't really know if they don't really know anything to start with ?

Won't be long, methinks, before it will get very difficult to verify many things, certainly current events. We're going to become true sheep. Genealogy is the least of our worries.

Sorry - bad John made me have another glass of wine and make this post.

Your concerns are legitimate, @John Winn. In some ways I'm glad that I can live out the rest of my life Happy that I lived when I did. My concerns are similar to yours, and I fear only for the younger generations in my family. May God bless them to be prosperous. Drink Up @John Winn!

 

[John Winn]

IF I had access to an AI tool that I trusted not to be hacking me, I would ask it a question about several generations of a family line for which I already KNEW the answers, to check the AI tool's ability to get things right. IF such a TRUSTED AI tool got the questioned generations and their facts right, then I would try using it to discover facts on some partially known and related unknown generations. (I would still be very wary of the results.)

True. But in your example you 1) actually have facts - i.e. know something and 2) are able to actually do research without AI. Very soon, you'll be one of a small percentage and the rest will just assume AI (or some other 'source') is correct because they don't have a clue. Geez, lots of kids these days can't read a clock or count change; they depend on a machine to tell them.

Really scary. But I've only got maybe ten or, at most, fifteen years so will escape many of the consequences. I can only imagine what the future will be like for somebody in their twenties.

 

[CWdiaryPA]

True. But in your example you 1) actually have facts - i.e. know something and 2) are able to actually do research without AI. Very soon, you'll be one of a small percentage and the rest will just assume AI (or some other 'source') is correct because they don't have a clue. Geez, lots of kids these days can't read a clock or count change; they depend on a machine to tell them.

Really scary. But I've only got maybe ten or, at most, fifteen years so will escape many of the consequences. I can only imagine what the future will be like for somebody in their twenties.

I hear you on the gratitude at missing what's coming. In so many ways our generation (give or take a decade) had it smooth.

Unfortunately, I believe I'll make it to about 93, so 30+ years. But one thing I AM certain of: every generation has the awake ones. Each college class I taught had that handful of kids alert, smart, with it. When the radio came along, when the idiot box got invented.... each new technology brought a rending of garments about future gens going to hell in a hand basket.

But I have faith.

There's going to be about the same # of walking idiots the human race has always displayed. Yet there'll still exist that handful in any group who keep the world spinning on its axis. We see it here at CWT, a few young ones. Just different challenges is all. Same as the old boss, no?

 

[Stryker65]

I fear that in a very short time this kind of "tool" will just be accepted by an ignorant, lazy society as providing a quick, truthful, answer to many things. As most of the masses already only 'know' what they see on YouTube how would they know they don't really know if they don't really know anything to start with ?

Won't be long, methinks, before it will get very difficult to verify many things, certainly current events. We're going to become true sheep. Genealogy is the least of our worries.

Sorry - bad John made me have another glass of wine and make this post.

I agree with Bad John's ideas, though. ChatGPT and other "Generative" AI does not often provide a truthful answer, although it IS quick. As to difficulties in verifying current events, I think that we can all agree, without going into specifics, that the "fake news" allegations thrown around against every news station has already prompted that.

True. But in your example you 1) actually have facts - i.e. know something and 2) are able to actually do research without AI. Very soon, you'll be one of a small percentage and the rest will just assume AI (or some other 'source') is correct because they don't have a clue. Geez, lots of kids these days can't read a clock or count change; they depend on a machine to tell them.

Really scary. But I've only got maybe ten or, at most, fifteen years so will escape many of the consequences. I can only imagine what the future will be like for somebody in their twenties.

As someone in their teens, I completely agree. Especially as I'm planning to go into academia for history, AI's going to be the death of me. Most people don't realize that ChatGPT, Grok, Gemini, etc are GENERATIVE, not analytical. Generative AI generates its information through making assumptions and not through gathering facts. Because who needs facts? As you said, many people who use Generative AI want FAST answers, with historical/verifiable accuracy second.

 

[John Winn]

I agree with Bad John's ideas, though. ChatGPT and other "Generative" AI does not often provide a truthful answer, although it IS quick. As to difficulties in verifying current events, I think that we can all agree, without going into specifics, that the "fake news" allegations thrown around against every news station has already prompted that.

As someone in their teens, I completely agree. Especially as I'm planning to go into academia for history, AI's going to be the death of me. Most people don't realize that ChatGPT, Grok, Gemini, etc are GENERATIVE, not analytical. Generative AI generates its information through making assumptions and not through gathering facts. Because who needs facts? As you said, many people who use Generative AI want FAST answers, with historical/verifiable accuracy second.

Ah, there's hope yet. My best to you. You are needed.

 

[CWdiaryPA]

Ah, there's hope yet. My best to you. You are needed.

He's that handful cohort I'm talking about.

 

[Gary Morgan]

There's a bonehead on another board that posted an AI generated image which he claimed "Is what my ancestor looked like." He uploaded photos of two siblings and it "combined" them and put the result in a cartoonish Civil War uniform. The sheep on the board all started asking how to do the same. I piped up and pointed out that that ISN'T what his CW ancestor looked like. If you combined the features of my two brothers, you wouldn't get anything close to what I look like. Unless you have an actual image of your ancestor, you don't have an image of your ancestor.

At one point, the AI that pops up when you run a google search now said that Henry Wirz was a POW at Andersonville. That didn't even put him on the right side in the war. (He was the commandant of Andersonville).

 

[rvbgrif]

I don't know why it is called "Intelligence". Its value is in its quick reach, not in its ability of determination.

The thing about "intelligence" is that there are two ways to make machines intelligence. You can write code, or you can use data.

With the AI revolution, we have found ways to combine code with massive data sets and make machines mimic human intelligence more closely... but it is a game of pulling data out of a big box of glop.

Data works way better for some questions like images. If you, say, asked a machine to identify a dog, you would have to write instructions on what makes a dog different from all other animals and objects with a high level of accuracy... not easy to do. But if you gave a machine pictures of dogs and told it to match a picture with a high level of accuracy, suddenly it becomes way smarter...

So somewhere, the machine found mayflower data in its glop box, and spat it out.

You are right, it has data but doesn't know how to decipher data because it is an oddly specific question. "Tell me about the Smith family of Pennsylvania". It is just going to google smith, pull up data, and spit it out. It doesn't know how to interpret her family or who is in her family or anything. Because the developer didn't write that... because that is an oddly specific question.. It is like asking a child to google "Smith, Pennsylvania" and seeing what it finds. The child will do its best and tell you what it sees... It... could be helpful, but you still have to verify.

 

[Stryker65]

Data works way better for some questions like images. If you, say, asked a machine to identify a dog, you would have to write instructions on what makes a dog different from all other animals and objects with a high level of accuracy... not easy to do. But if you gave a machine pictures of dogs and told it to match a picture with a high level of accuracy, suddenly it becomes way smarter...

Hence the "Are You A Robot? Choose All Images With A Dog" safety measures on different websites -- I've seen it especially on Reddit.

 

[rvbgrif]

Hence the "Are You A Robot? Choose All Images With A Dog" safety measures on different websites -- I've seen it especially on Reddit.

The way those work, is that you pick out the pictures with a dog, and the machine stores all the answers from everyone, and uses them to identify dogs in pictures by scanning and comparing.

Its easy to pick out machines because not only do they have trouble identifying images... they have trouble scrolling. Either they scroll in a perfectly straight line, or they teleport the cursor to the middle of the image and click. Humans will move the mouse in a wavy line and work our way over to a random part of the image and click.

 

[13 IL Cav & 64 GA Inf]

There's a bonehead on another board that posted an AI generated image which he claimed "Is what my ancestor looked like." He uploaded photos of two siblings and it "combined" them and put the result in a cartoonish Civil War uniform. The sheep on the board all started asking how to do the same. I piped up and pointed out that that ISN'T what his CW ancestor looked like. If you combined the features of my two brothers, you wouldn't get anything close to what I look like. Unless you have an actual image of your ancestor, you don't have an image of your ancestor.

At one point, the AI that pops up when you run a google search now said that Henry Wirz was a POW at Andersonville. That didn't even put him on the right side in the war. (He was the commandant of Andersonville).

I have zero confidence in AI in its current stage. It is only as good as the data put into it (needs to be trillions of data items), the programmed logic, and the application of the programmed logic to the data. AI is in its infancy. A baby can't tell you much.

 

[lupaglupa]

I do think that over time AI will improve and become a useful tool. Hopefully people will use it with full knowledge of its limitations until then.

 

[The Ninth Ohio]

I do think that over time AI will improve and become a useful tool. Hopefully people will use it with full knowledge of its limitations until then.

It already is a useful tool. The issue is people are asking it to do things it was never designed for, and then using its failure to beat them at their guessing game as proof it's not useful.

"AI is unsuccessful because it didn't get the date of my 4th great grandpa's promotion from musician third class to musician second class" doesn't really prove anything to us about AI programs. If anything it says more about the users' lack of understanding the purpose and capabilities of large language models.

 

[Gary Morgan]

It already is a useful tool. The issue is people are asking it to do things it was never designed for, and then using its failure to beat them at their guessing game as proof it's not useful.

My students are finding it very useful in writing their papers for them. Happily, it doesn't annotate sources yet.

 

[13 IL Cav & 64 GA Inf]

It already is a useful tool. The issue is people are asking it to do things it was never designed for, and then using its failure to beat them at their guessing game as proof it's not useful.

"AI is unsuccessful because it didn't get the date of my 4th great grandpa's promotion from musician third class to musician second class" doesn't really prove anything to us about AI programs. If anything it says more about the users' lack of understanding the purpose and capabilities of large language models.

I don't think that many people are asking for that level of detail. You are correct in that different AI programs will be designed for different tasks, and different data loads will be used for example between programs for medicine and those for history, except for the overlaps between the two.

 

[The Ninth Ohio]

My students are finding it very useful in writing their papers for them. Happily, it doesn't annotate sources yet.

If you know what you're doing, you can make an AI response look very close to one created by a human, and those who aren't as familiar with the subject are at a tremendous disadvantage to catch it.

Unfortunately I know of two people in the last two years who earned an MA using AI. Academia in general and history specifically are completely unprepared for what's coming.

 

[CWdiaryPA]

If you know what you're doing, you can make an AI response look very close to one created by a human, and those who aren't as familiar with the subject are at a tremendous disadvantage to catch it.

Unfortunately I know of two people in the last two years who earned an MA using AI. Academia in general and history specifically are completely unprepared for what's coming.

I wonder if there's a way to catch it by prompts. Because we (instructors) used Turnitin at universities if we got suspicious. Looks like the company is evolving to catching AI plagiarism with a system of detection: https://www.turnitin.com/solutions/topics/ai-writing/