One of the main reasons for this effort was the need to have credible, accessible information to be used to contend with legal actions brought by Palestinian residents, human rights organizations and leftist movements challenging the legality of construction in the settlements and the use of private lands for establishing or expanding them. The painstakingly amassed data was considered political dynamite.

The defense establishment, led by Defense Minister Ehud Barak, steadfastly refused to publicize the figures, arguing, for one thing, that doing so could endanger state security or harm Israel's foreign relations.

Someone who is liable to be particularly interested in the data collected by Spiegel is George Mitchell, President Barack Obama's special envoy to the Middle East, who arrived in Israel last week for his first visit since his appointment. It was Mitchell who authored the 2001 report that led to the formulation of the road map, which established a parallel between halting terror and cessation of construction in the settlements.

The information in the official database, the most comprehensive one ever compiled in Israel about the territories, was recently obtained by Haaretz. An analysis of the data reveals that, in the vast majority of the settlements - about 75 percent - construction, sometimes on a large scale, has been carried out without the appropriate permits or contrary to permits that were issued. The database also shows that, in more than 30 settlements, extensive construction of buildings and infrastructure (roads, schools, synagogues, yeshivas and even police stations) has been carried out on private lands belonging to Palestinian West Bank residents.

The data, it should be stressed, does not refer only to the illegal outposts (information about which was included in the well-known report authored by attorney Talia Sasson and published in March 2005), but to the very heart of the settlement enterprise. It includes veteran ideological settlements like Alon Shvut (established in 1970 and currently home to 3,291 residents); Ofra (established in 1975, home to 2,708 residents, including former Yesha Council spokesman Yehoshua Mor Yosef and media personalities Uri Elitzur and Hagai Segal); and Beit El (established in 1977, population 5,308, including Hagai Ben-Artzi, brother of Sara Netanyahu). Also included are large settlements founded primarily due to economic motives, such as the city of Modi'in Ilit (established in 1990 and now home to 36,282 people), or Givat Ze'ev outside Jerusalem (founded in 1983, population 11,139), and smaller settlements such as Nokdim near Herodion (established in 1982, population 851, including MK Avigdor Lieberman).

The information contained in the database does not conform to the state's official position, as presented, for instance, on the Foreign Ministry Web site, which states: "Israel's actions relating to the use and allocation of land under its administration are all taken with strict consideration of the rules and norms of international law ... Israel does not requisition private land for the establishment of settlements."

Since in many of the settlements, it was the government itself, primarily through the Construction and Housing Ministry, that was responsible for construction, and since many of the building violations involve infrastructure, roads, public buildings and so on, the official data also shows government responsibility for the unrestrained planning and lack of enforcement of regulations in the territories. The extent of building violations also attests to the poor functioning of the Civil Administration, the body in charge of permits and supervision of construction in the territories.

According to the 2008 data from the Central Bureau of Statistics, approximately 290,000 Jews live in the 120 official settlements and dozens of outposts established throughout the West Bank over the past 41 years.

"Nothing was done secretively," says Pinchas Wallerstein, director general of the Yesha Council of settlements and a leading figure in the settlement project. "I'm not familiar with any [building] plans that were not the initiative of the Israeli government." He notes that if the owners of private land upon which settlements are built were to complain and the court were to accept their complaint, then the structures would have to be moved somewhere else. "This has been the Yesha Council's position for the past years," he adds.

You would never know it from touring several of the settlements in which massive construction has taken place on private Palestinian land. Entire neighborhoods built without permits or on private property are inseparable parts of the settlements. The sense of dissonance only intensifies when you find that municipal offices, police and fire stations were also built upon and currently operate on lands that belong to Palestinians.

On Mishkenot Haro'im Street in the Kokhav Yaakov settlement, a young mother is carrying her two children home. "I've lived here for six years," she says, sounding surprised when informed that her entire neighborhood was built upon private Palestinian land. "I know there's some small area in the community that is disputed, but I never heard that this is private land." Would she have built her home on this land had she known this from the start? "No," she answers. "I wouldn't have kicked anyone out of his home."

Not far away, at the settlement's large and unkempt trailer site, which is also built on private land, a young newlywed couple is walking to the bus stop: 21-year-old Aharon and his 19-year-old wife, Elisheva. They speak nearly perfect Hebrew, despite having grown up in the United States and having settled permanently in Israel just a few months ago, after Aharon completed his army service in the ultra-Orthodox Nahal Brigade unit. Now he is studying computers at Machon Lev in Jerusalem. Asked why they chose to live here of all places, they list three reasons: It's close to Jerusalem, it's cheap and it's in the territories. In that order.

The couple pay their rent, NIS 550 a month, to the settlement secretariat. As new immigrants, they are still exempt from having to pay arnona (municipal tax). Aharon doesn't seem upset upon learning that his trailer sits on private land. It doesn't really interest him, he says: "I don't care what the state says - the Torah says that the entire Land of Israel is ours." And what will happen if they're told to move to non-private land? "We'll move," he says without hesitation.

Even today, more than two years after concluding his official role, Baruch Spiegel remains loyal to the establishment. In a conversation, he notes several times that he signed a confidentiality agreement and is thus unwilling to go into the details of the work for which he was responsible. He was appointed to handle several issues about which Israel had given a commitment to the United States, including improving conditions for Palestinians whose lives were adversely affected by the separation fence, and supervising Israel Defense Forces soldiers at the checkpoints.

Two years ago, Haaretz reporter Amos Harel revealed that Spiegel's main task was to establish and maintain an up-to-date database on the settlement enterprise. This was after it became apparent that the United States, as well as the settlement monitoring team of the Peace Now organization, was already in possession of much more precise information about settlement construction than the defense establishment.

Spiegel's database contains written documents backed up by aerial photos and Geographic Information Systems (GIS) data that includes information on the status of the land and the official boundaries of each settlement.

"The work took two and a half years to complete," says Spiegel. "It was done in order to check the status of the settlements and the outposts, and to achieve the greatest possible accuracy in terms of the database - the status of the property, the legal status, the sector boundaries, municipal building plans, government decisions, lands whose ownership is unclear. It was full-time, professional work, carried out by a professional team of legal experts, planning people, GIS experts. And I hope this work continues, because it is very vital. One has to know what's going on there and make decisions accordingly."

Who is keeping track of all this now?

Spiegel: "I suppose it's the Civil Administration."

Why was there no database like this before your appointment?

"I don't know how much of a focus there was on it."

Why do you think the state is not publicizing the data?

"It's a sensitive and complex subject and there are all kinds of considerations, political and security-related. There were questions about the public's right to know, the freedom of information law. You should ask the officials in charge."

What are the sensitive matters?

"It's no secret that there are violations, that there are problems related to land. It's a complicated problem."

Is there also a problem for the country's image?

"I didn't concern myself with image. I was engaged in Sisyphean work to ensure that, first of all, they'll know what exists, what's legal and what's not, and what the extent of the illegality is - whether it involves the takeover of private Palestinian land or is in the process of obtaining proper building permits. Our job was to do the meticulous work of examining all the settlements and outposts that existed then ... We found what we found and passed it on."

Do you think this information should be published?

"I think they've already decided to publish the simpler part, concerning areas of jurisdiction. There are things that are more sensitive. It's no secret that there are problems, and it's impossible to do something illegal and say it's legal. I can't elaborate, because I'm still bound to maintain confidentiality."

Says Dror Etkes, formerly the coordinator of Peace Now's settlement-monitoring project and currently director of the Land Advocacy Project for the Yesh Din organization: "The government's ongoing refusal to reveal this material on the pretext of security reasons is yet another striking example of the way in which the state exploits its authority to reduce the information at the citizens' disposal, when they wish to formulate intelligent positions based on facts, rather than lies and half-truths."

Following the initial exposure of the material, the Movement for Freedom of Information and Peace Now requested that the Defense Ministry publish the database information, in accordance with the Freedom of Information Law. The ministry refused.

"This is a computerized database that includes detailed information, in different cross-sections, regarding the Jewish settlements in Judea and Samaria," the ministry said in response. "The material was collected by the defense establishment for its purposes and includes sensitive information. The ministry was asked to allow a review of the material in accordance with the Freedom of Information Law, and after consideration of the request, decided not to hand over the material. The matter is pending and is the subject of a petition before the Administrative Affairs Court in Tel Aviv."

The database provides an alphabetical survey of settlements. For each entry, it notes the source of the settlement's name and the form of settlement (urban community, local council, moshav, kibbutz, etc.); its organizational affiliation (Herut, Amana, Takam, etc.); the number of inhabitants; pertinent government decisions; the official bodies to which the land was given; the status of the land upon which the settlement was built (state land, private Palestinian or Jewish land, etc.); the illegal outposts built in proximity to the settlement; and the extent to which valid building plans have been executed. Beneath each entry, highlighted in red, is information on the extent of construction that has been carried out without permission and its exact location in the settlement.

Among all the revelations in the official data, it's quite fascinating to see what was written about Ofra, a veteran Gush Emunim settlement. According to a recent report by the B'Tselem human rights organization, most of that settlement's developed area sits on private Palestinian land and therefore falls into the category of an illegal outpost that is supposed to be evacuated. The Yesha Council responded to that report, saying that the "facts" in it are "completely baseless and designed to present a false picture. The inhabitants of Ofra are careful to respect the rights of the Arab landowners, with whom they reached an agreement regarding the construction of the neighborhoods, as well as an agreement that enables the private landowners to continue to work their lands."

But the information on Ofra contained in the database leaves no room for doubt: "The settlement does not conform to valid building plans. A majority of the construction in the community is on registered private lands, without any legal basis whatsoever and no possibility of [converting the land to non-private use]."

The database also gives a detailed description of where construction was carried out in Ofra without permits, "in the original part of the settlement - more than 200 permanent residential structures, agricultural structures, public structures, lots, roads and orchards (in regard to which Plan 221 was submitted, but not advanced due to a problem of ownership)."

Yesha Council chairman Danny Dayan responds: "I am not familiar with that data."

Another place where the data reveals illegal construction is Elon Moreh, one of the most famous settlements in the territories. In June 1979, several residents of the village of Rujib, southeast of Nablus, petitioned the High Court, asking it to annul the appropriation order for 5,000 dunams (1,250 acres) of land in their possession, which had been designated for the construction of the settlement. In court, the government argued, as it did regularly at the time, that the construction of the settlement was required for military purposes, and therefore the appropriation orders were legal. But in a statement on behalf of the petitioners, former chief of staff Haim Bar-Lev asserted then that, "In my best professional judgment, Elon Moreh does not contribute to Israel's security."

The High Court of Justice, relying on this statement and information from the original core group of settlers of Elon Moreh - who also argued that a temporary settlement was not established for security purposes, and that a permanent one was built instead - instructed the IDF to evacuate it and return the lands to their owners. The immediate consequence of the ruling was to find an alternative site for the settlement, on property previously defined as "state lands." Following this ruling, Israel stopped officially using military injunctions in the territories for the purpose of establishing new settlements.

The lands that were originally appropriated for the purpose of building Elon Moreh were returned to their Palestinian owners, but according to the database, in Har Kabir, the settlement's new site, too, "most of the construction was carried out without approved, detailed plans, and some of the construction involved trespassing on private lands. As for the state lands in the settlement, a detailed plan, No. 107/1, was prepared and published on 16/7/99, but has yet to go into effect."

The Shomron Regional Council, which includes Elon Moreh, says in response: "All the neighborhoods in the settlement were planned, and some were also built, by the State of Israel through the Housing and Construction Ministry. The residents of Elon Moreh did not trespass at all and any allegation of this kind is also false. The State of Israel is tasked with promoting and approving the building plans in the settlement, as everywhere else in the country, and as for the plans that supposedly have yet to receive final validity, just like many other communities throughout Israel, where processes continue for decades, this does not delay the plans, even if the planning is not complete or carried out in tandem."

Beit El, another veteran settlement, was also, according to the database, established "on private lands seized for military purposes (in fact, the settlement was expanded onto private lands, by trespassing in the northern section of the settlement), on state lands that were appropriated during the Jordanian period (the Maoz Tzur neighborhood in the southern part of Beit El)."

According to the official data, in the absence of approved plans, construction in Beit El includes the local council's offices and the "northern neighborhood (Beit El Bet) that was built for the most part on private lands. The neighborhood comprises widespread construction, public buildings and new ring roads."

Moshe Rosenbaum, head of the Beit El local council, responds: "Unfortunately, you are cooperating with the worst of Israel's enemies and causing tremendous damage to the whole country."

Ron Nahman, mayor of Ariel, was re-elected to a sixth term in the last elections. Nahman is a long-time resident of the territories and runs a fascinating heterogeneous city. Between a visit to the trailer site where evacuees from Netzarim are housed, and a stop at a shop that sells pork and other non-kosher products - mostly to the city's large Russian population - he complains about the cessation of construction in his city and about his battles with the Civil Administration over every building permit.

Ariel College, Nahman's pride and joy, is also mentioned in the database: "The area upon which Ariel College was built was not regulated in terms of planning." It further explains that the institution sits on two separate plots, and the new plan has not yet been discussed.

Nahman confirms this, but notes that the planning issue was recently resolved. When told that dozens of settlements include areas that were constructed on private lands, he is not surprised. "That's possible," he says. The fact that in three-quarters of the settlements there has been construction that deviates from the approved plans doesn't surprise him either.

"All the complaints should be directed at the government, not at us," he explains. "As for the small and communal settlements, they were planned by the Housing Ministry's rural construction administration. The larger communities are planned by the ministry's district offices. It's all the government. Sometimes the Housing Ministry is responsible for budgetary construction, which is construction covered by the state budget. In the Build Your Own Home program, the state pays a share of the development costs and the rest is paid for by the individual. All these things are one giant bluff. Am I the one who planned the settlements? It was [Ariel] Sharon, [Shimon] Peres, [Yitzhak] Rabin, Golda [Meir], [Moshe] Dayan."

Most of the territories of the West Bank have not been annexed to Israel, and therefore regulations for the establishment and construction of communities there differ from those that apply within Israel proper. The Sasson report, which dealt with the illegal outposts, was based in part on data collected by Spiegel, and listed the criteria for the establishment of a new settlement in the territories:

1. The Israeli government has to issue a decision to establish the settlement.

2. The settlement has to have a defined jurisdictional area.

3. The settlement has to have a detailed, approved plan.

4. The settlement has to be located on state land or on land purchased by Israelis and registered under their name in the Land Registry.

According to the database, the state gave the World Zionist Organization (WZO) and/or the Housing Ministry authorization to plan and build on most of the lands on which the settlements were constructed. These bodies allocated the property to the people who eventually carried out the actual settlement construction: Sometimes it was WZO's Settlement Division; other times it was the Housing Ministry itself, sometimes through the Rural Building Administration. In several cases, settlements were built by Amana, the settlement arm of the Gush Emunim organization. Another body cited in the database as having received allocations and as being responsible for construction in some settlements is Gush Emunim's Settler National Fund.

Regular state schools and religious schools (Talmudei Torah) have also been built on Palestinian lands. According to the database, in the southern part of the Ateret settlement, for instance, "15 structures were built outside of state lands, which are used for the Kinor David yeshiva. There are also new ring roads and a special security area that is illegal." The sign at the entrance says the yeshiva was built by the Amana settlement movement, the Mateh Binyamin local council and the WZO settlement division.

In the Psagot settlement, where there has also been a lot of construction on private land, it's easy to discern the terraced design typical of Palestinian agriculture in the region. According to the database, in Psagot there are "agricultural structures (a winery and storehouses) to the east of the settlement, close to the grapevines cultivated by the settlement, via trespassing."

During a visit there, the winery was found to be abandoned. Its owner, Yaakov Berg, acquired land from the Israel Lands Administration near the Migron outpost and a new winery and regional visitors' center is currently under construction there.

"The vineyards are located in Psagot," says Berg, who is busy with preparing the new site. From the unfinished observation deck, one can see an enormous quarry in the mountains across the way.

"If I built a bathroom here without permission from the Civil Administration, within 15 minutes, a helicopter would be here and I'd be told that it was prohibited," Berg complains. "And right here there's an illegal Palestinian quarry that continues to operate."