Israel's State Archives to Use Artificial Intelligence to Release Millions of Documents

The technology will locate classified information and enable archives to examine documents quickly – without it, the process would take thousands of years

Ofer Aderet
Ofer Aderet
Send in e-mailSend in e-mail
The process of scanning documents from the State Archives to make digital copies at a factory in Petah Tikva, May 18, 2016.
The process of scanning documents from the State Archives to make digital copies at a factory in Petah Tikva, May 18, 2016.Credit: Dudu Bachar
Ofer Aderet
Ofer Aderet

The Israel State Archives will use artificial intelligence technology to locate classified information in its documents, in a program that Chief State Archivist Ruti Abramovitz said on Wednesday would save thousands of years of work to release the documents to the public. The documents will also be checked by experts to determine whether they can be published.

At the annual conference of the Association of Israeli Archivists, Abramovitz revealed that the archives’ plan to continue their “digital revolution” that was started by her predecessor, Dr. Yaacov Lozowick. As part of this revolution, hundreds of millions of documents in the State Archives have been scanned over the past few years, and have been transferred from paper into digital copies. But even after the documents are scanned, the archivists cannot release them to the public immediately, because some of them contain classified information and have to be examined before their release.

LISTEN: Kosovo, COVID and Bibi's brilliant bravado

0:00
-- : --

“We researched the market, studied what technology companies know … and found that the best method [for making archival material accessible to the public] is artificial intelligence,” she said. According to the plan, a computerized artificial intelligence system that “speaks” Hebrew would read the hundreds of millions of documents preserved in the State Archive by itself – a task that the limited team at the archives today would be unable to do on its own – and know how to locate classified security information, or sensitive personal information, that now prevents the documents from being released. This information would be passed on to the experts, who would decide whether it is possible to release them and under what conditions. The State Archives’ staff would “teach” the system their professional secrets and it would do the work in their place, said Abramovitz.

Artificial intelligence is a broad term, which refers to tools, methods and ways to give a computer similar abilities to those of people, including that of learning.

Documents from the State Archives being scanned to make digital copies in Petah Tikva, May 18, 2020.Credit: Dudu Bachar

The State Archives are now in contact with 10 potential companies in preparation for issuing a tender on the process. Abramovitz said that national archives exist around the world that use artificial intelligence systems to release materials, but it is done on a small and limited basis. “We want to make total use of the system, and hope to be the international guiding light for this project,” she said.

Abramovitz referred to another challenge facing the State Archives in the digital era: The documents, which in the past were written and printed on paper, are created digitally today, including through the use of email, WhatsApp and other apps and computer programs, which are never even printed out. The State Archives are still looking for technology that will allow them to receive these digital materials, even though government ministries have been using them for decades to produce digital materials that don’t exist on paper, she said. “Just this year, we’ve begun to accept digital materials. Our systems don’t know how to do it very well. We are trying to understand how to do it,” she added.

Without artificial intelligence, it will be impossible to cope with the amount of material the digital era will supply to the archives – an amount many times greater than that of the paper documents they have. But Abramovitz said that now, when the material is digital, it is possible to preserve everything. This is in contrast to the present situation, in which a large portion documents are destroyed because there is no need for or interest in them. “An enormous and endless opportunity for research will open up here,” she said.

In order to receive archival documents, the public can enter the State Archives’ website and submit a request for documents to be scanned so they can be uploaded to the website. But Abramovitz admits that digitization of the archives still does not enable researchers to make optimal use of them, in part because it takes a long time from when the request for the material to be scanned is made, which users can submit from their own computer, to when it is uploaded to the site.

Documents about the missing Yemenite children affair at the Israel State Archives, 2016.Credit: Ofer Aderet

Abramovitz also discovered that tens of thousands of containers, which hold archival materials, are stuck in a bottleneck and have yet to reach the State Archives because there is no place to store them. The plan to build a new warehouse for the archives will be restarted in the next few years, allowing them to accept the materials that government ministries want to deposit, but are currently unable to.

In addition, the preparation of a new “Archives Law” is in the process of being finalised. The new law is designed to bring the antiquated and irrelevant existing law up to date and adapt it to the present era. Abramovitz added that the State Archives’ website, which was taken offline after it was hacked, will go back online on Wednesday.

Abramovitz also promised that the reading room at the archives will reopen by January, after it was closed three years ago as part of the move to digitize the archives. But now it will only be accessible to researchers and not to the general public. The decision was made after an academic protest against the closure, along with a fear at the State Archives that they had “sentenced [themselves] to an almost total death sentence concerning both new and veteran researchers.” Abramovitz said that at the State Archives realized that if they did not reopen the reading room, researchers might very well give up in advance on academic research that requires the use of materials located in the State Archives.

Comments