Using AI To Transcribe Handwriting

[Stryker65]

Thought I'd make this post here to talk about how I use AI in my research. Google, a year ago, released a tool called Notebook LM, to which one can add various PDFs/images/other media as sources, and the AI can analyze and disseminate the information.

As some of you know, I've been working for a few years now on a project to uncover the backstories of all 595 USCT field officers. Notebook LM has helped me greatly with researching some of the lesser known ones; in this example we will go with a man by the name of Simon Jones, the colonel of the 93rd USCT. Cursory research indicated that Simon Jones was mustered in as a colonel and mustered out as a colonel. There was almost no way to determine his previous military experience, since Simon Jones is a relatively common name. However, utilizing Fold3, I found his CMSR, and with it, I found some letters he had written.

Letter #1:

Now, I could transcribe this myself, as thousands of historians have done with similar material over the past five hundred years. But, I could also put this into NotebookLM, which I did:

Adding Sources Into NotebookLM

Then, using the Chat Feature, I gave the AI the prompt to "Transcribe the letter," and within seconds it returned:

"Transcribe the letter"

Pretty cool, isn't it? I repeated this five additional times, with five additional letters.

Five additional letters

Finally, when I had found enough information, I asked the AI to summarize the key points:

In summary:

In conclusion: Through analysis of these six letters, I determined that Simon Jones was a Unionist Texan lawyer who fled to New Orleans, inherited a large estate and many slaves, and repeatedly cajoled Union authorities for a commission to raise troops. As it turned out, the author of the Dwight letter (Brig. Gen. Lorenzo Thomas) was correct -- Jones was dismissed by an examining board two years later, along with half the officers in his regiment, and the men reassigned.

In conclusion with regards to AI use in research: This use of AI, I think, is one of the only truly beneficial ones. The AI can still make mistakes, yes, but this AI is specially trained -- it is NOT Generative AI; rather, it is limited only to what sources are provided to it. I can see problems with how this AI could be used to cheat in essay writing classes, but from discussions with some of my professors last year, there are already plans in motion to circumvent this strategy.

 

[A. Roy]

This looks amazing! I can see this as useful for raw transcribing, which I can do but which is time-consuming. Have you confirmed the accuracy of the transcriptions?

Am I correct that Google has rebranded this product as Gemini Notebook?

AR

 

[Aaron Vinson]

Transkribus is pretty good - https://www.transkribus.org/
not always perfect but will help when you are stuck, even if it only gets one letter right (which can sometimes help you with the whole word)

 

[gary]

Wow! Finally a legitimate use of AI. I remember it taking me over an hour to transcribe some things and sometimes it was only because of the words around the mystery word.

 

[Stryker65]

This looks amazing! I can see this as useful for raw transcribing, which I can do but which is time-consuming. Have you confirmed the accuracy of the transcriptions?

I have confirmed the transcriptions. Somehow, it's much easier to read when I think I know what it says!

Am I correct that Google has rebranded this product as Gemini Notebook?

You're right! I hadn't seen that, but after looking it up it is true.

 

[Josh The Lighthouse Guy]

Transcribing handwriting is one of the best uses for pattern recognition AI.

Success depends on not only the quality of the handwriting but also the quality of the scanned document. Any numbers not spelled out especially need careful checking. This per a professional conference I attended earlier this year.

 

[huskerblitz]

We've discussed this before on here, but yes, AI can really be helpful for transcribing text. May not be 100% perfect, but should be enough to get you the correct word(s) it misses on. One of the best uses of AI I have yet found.

 

[DaveBrt]

Transkribus is pretty good - https://www.transkribus.org/
not always perfect but will help when you are stuck, even if it only gets one letter right (which can sometimes help you with the whole word)

I tried Transkribus on 5 letters. On lines of text I was able to read clearly and promptly, it gave gibberish. It tried french on a couple of lines and German on another. I signed up for the free account and got even worse results.

 

[Aaron Vinson]

I tried Transkribus on 5 letters. On lines of text I was able to read clearly and promptly, it gave gibberish. It tried french on a couple of lines and German on another. I signed up for the free account and got even worse results.

like I said sometimes it is *** but sometimes helps a lot, I try to use my zoomer brain most of the time since they didn't teach us cursive in school but when i'm really stuck I'll give it a go.

 

[Belfoured]

We've discussed this before on here, but yes, AI can really be helpful for transcribing text. May not be 100% perfect, but should be enough to get you the correct word(s) it misses on. One of the best uses of AI I have yet found.

That's a great point. One sharp contrast is the reliance of attorneys on AI in litigation. I do a lot of appellate work and AI apps consistently fall short in doing the type of sophisticated analysis of SCOTUS and other appellate decisions that is required. There are several cases just this year in which judges have hammered attorneys for filing documents that contain "hallucinated" citations. At most it might point you in the direction where to start actual research but that's about it.

 

[Josh The Lighthouse Guy]

One sharp contrast is the reliance of attorneys on AI in litigation. I do a lot of appellate work and AI apps consistently fall short in doing the type of sophisticated analysis of SCOTUS and other appellate decisions that is required. There are several cases just this year in which judges have hammered attorneys for filing documents that contain "hallucinated" citations. At most it might point you in the direction where to start actual research but that's about it.

AI could be helpful finding legal cases an attorney wasn't aware of - essentially an advanced search engine. "Give me every legal case in the State of Montana that cited Green v. Bandersnatch."

Maybe it would do better if very strictly trained specialty AI, using only the contents of a law library. Ultimately, LLMs are about pattern recognition, not constructing careful logical arguments. Any AI that includes internet training will be awash with all sorts of spurious and ignorant interpretations of various legal rights, not to mention fictional legal cases from fictional properties.

 

[Belfoured]

AI could be helpful finding legal cases an attorney wasn't aware of - essentially an advanced search engine. "Give me every legal case in the State of Montana that cited Green v. Bandersnatch."

Maybe it would do better if very strictly trained specialty AI, using only the contents of a law library. Ultimately, LLMs are about pattern recognition, not constructing careful logical arguments. Any AI that includes internet training will be awash with all sorts of spurious and ignorant interpretations of various legal rights, not to mention fictional legal cases from fictional properties.

I agree on finding possible cases - that was my point about giving a start on the research. In fact, I've used it solely for that but even then only as a backup/supplement. Any lawyer who doesn't actually read the decisions is incompetent. AI can give me "holdings", etc but that's usually meaningless in applying those to a specific case.

 

[bschulte]

I'm doing something similar with ordnance officers in the ANV during the Siege of Petersburg. I feed Claude 20 images at a time. It transcribes but asks me to verify various things, especially officer names, dollar amounts, ordnance store amounts, etc. I'm having it create short biographical accounts based on Krick's Staff Officers in Gray entries as well as their CMSR files at Fold3. I've attached an example of a completed man, Peyton L. Manning (no, not an ancestor of THAT Peyton Manning, Harry Smeltzer at Bull Runnings explored this.) IT even shows where I would question or correct the AI as part of the record.

 

[Library Lady]

That's a great point. One sharp contrast is the reliance of attorneys on AI in litigation. I do a lot of appellate work and AI apps consistently fall short in doing the type of sophisticated analysis of SCOTUS and other appellate decisions that is required. There are several cases just this year in which judges have hammered attorneys for filing documents that contain "hallucinated" citations. At most it might point you in the direction where to start actual research but that's about it.

Off on a siding for a minute, please. Do courts still use real live human court reporters/stenographers or has that gone strictly to speech recognition? If it's gone to speech recognition, how well does that really work? As a medical transcriptionist, I had to clean up "speech wreck" for about 20 years. It never seemed to get much better, and my own medical records still don't look that good. I saw that it was going into legal in hopes of getting rid of the court reporters/stenographers. I saw one set-up where there was still a person who wore a contraption that looked vaguely like the oxygen mask a pilot wears. This person had to repeat everything that was said in court. The mask apparently cut out extraneous noise, and the software was supposed to work better because it was trained to only that person's voice. Yeah, not my idea of a good time. Back to the mainline now.

 

[Belfoured]

Off on a siding for a minute, please. Do courts still use real live human court reporters/stenographers or has that gone strictly to speech recognition? If it's gone to speech recognition, how well does that really work? As a medical transcriptionist, I had to clean up "speech wreck" for about 20 years. It never seemed to get much better, and my own medical records still don't look that good. I saw that it was going into legal in hopes of getting rid of the court reporters/stenographers. I saw one set-up where there was still a person who wore a contraption that looked vaguely like the oxygen mask a pilot wears. This person had to repeat everything that was said in court. The mask apparently cut out extraneous noise, and the software was supposed to work better because it was trained to only that person's voice. Yeah, not my idea of a good time. Back to the mainline now.

Courts still use human reporters/stenographers at hearings but many do not, instead audio recording hearings for later transcription - such as needed for an appeal. That process, in turn, could involve speech recognition but it has its own shortcomings regarding background noise, identifying/differentiating speakers, dealing with accents/dialect, legal terminology, etc - plus the transcript must be certified by a human.

 

[Intrafinesse]

Thought I'd make this post here to talk about how I use AI in my research. Google, a year ago, released a tool called Notebook LM, to which one can add various PDFs/images/other media as sources, and the AI can analyze and disseminate the information.

As some of you know, I've been working for a few years now on a project to uncover the backstories of all 595 USCT field officers. Notebook LM has helped me greatly with researching some of the lesser known ones; in this example we will go with a man by the name of Simon Jones, the colonel of the 93rd USCT. Cursory research indicated that Simon Jones was mustered in as a colonel and mustered out as a colonel. There was almost no way to determine his previous military experience, since Simon Jones is a relatively common name. However, utilizing Fold3, I found his CMSR, and with it, I found some letters he had written.

Letter #1:

Now, I could transcribe this myself, as thousands of historians have done with similar material over the past five hundred years. But, I could also put this into NotebookLM, which I did:

Adding Sources Into NotebookLM


Then, using the Chat Feature, I gave the AI the prompt to "Transcribe the letter," and within seconds it returned:

"Transcribe the letter"


Pretty cool, isn't it? I repeated this five additional times, with five additional letters.

Five additional letters


Finally, when I had found enough information, I asked the AI to summarize the key points:

In summary:


In conclusion: Through analysis of these six letters, I determined that Simon Jones was a Unionist Texan lawyer who fled to New Orleans, inherited a large estate and many slaves, and repeatedly cajoled Union authorities for a commission to raise troops. As it turned out, the author of the Dwight letter (Brig. Gen. Lorenzo Thomas) was correct -- Jones was dismissed by an examining board two years later, along with half the officers in his regiment, and the men reassigned.

In conclusion with regards to AI use in research: This use of AI, I think, is one of the only truly beneficial ones. The AI can still make mistakes, yes, but this AI is specially trained -- it is NOT Generative AI; rather, it is limited only to what sources are provided to it. I can see problems with how this AI could be used to cheat in essay writing classes, but from discussions with some of my professors last year, there are already plans in motion to circumvent this strategy.

I think thats GREAT!
It's difficult to read many first hand accounts.
This makes them far more accessable and searchable.