Сообщения

The distributed search engine YaCy version 1.0

Изображение
Activists of the free software movement launched a new project: search engine YaCy , which in the future can compete with Google, Yahoo/Bing, Yandex and other closed commercial search engines. Version of YaCy 1.0, was released after five years of development, and now to join the network finally can anyone. YaCy search engine works on the principle of P2P, i.e. storing the search index and processing search queries are carried out not on a Central server and a distributed network of peers Freeworld. The network can log each user that installed the software under GNU/Linux, Windows or Mac OS, the network is now about a thousand nodes. network Diagram Freeworld peers in real time If you have your website, proindeksirujut if it and place the index in its own node Freeworld. To do this, after indexing disable function of the software Distribution DHT and DHT Receive. If you want to index all the interesting sites and use their local search engine ( screenshot ) A d...

Physics of high temperatures

All probably already noticed that in PostgreSQL 8.4 appeared the new mode database: Warm standby . When it during operation of the database Write-Ahead logs (WAL) are broadcast into the subject database where real-time apply, as if it had occurred on the primary system. Therefore, if the main database for some reason (lightning/tornado/third world war/other disaster) fails, it will be possible to instantly switch to a subordinate database (which will be quite relevant compared to the main base) and use it in the future. But, unfortunately, "warm stendbay" implies that the sub-system continuously occurs, the database recovery process; it follows that while the main base is alive, sub-base can not be used. If you read at least blog depesz (not to mention the commit logs), then you already know what I'm getting at; if not, then... the 19th of December last year in a development version of PostgreSQL 8.5 has been added Hot standby . Now, when setting up WAL repl...

The Twi Journal

Изображение
for a Long time did not dare to write on Habr. At least, due to technical instability of the project. Now that the work is organized (sincerely hope so), we got a small recognition in the form of grant from Yuri Milner and Pavel Durov, I'm ready to send the project to aromacology. My name is Nikita Likhachev, I want to tell you about The Twi Journal . This newspaper, which is based on automatic analysis of Russian-speaking Twitter. the project Idea To construct a robot that can analyze the air Russian-speaking segments of the networks Twitter, Instagram and Foursquare. Then output this content in a convenient form on one site and to diversify its placement to send to other social networks. Someone I wonder what's happening on Twitter, but he is reluctant to leave Vkontakte or Facebook . And someone just a little time to follow everything he wants for ten minutes to assess the agenda. the sample objective Project in any case does not claim to absolute o...

Search engine YossarianLives looks for metaphors

As a rule, search on the Internet aimed at obtaining some factual information — as is done in universal Google or Yandex, or in a number of specialized search engines: restaurants, tickets, coupons, torrent, etc. / > This year the company YossarianLives intends to launch a search engine "metaphors" ( metaphorical search engine ), having a sponsor Deutsche Bank and the Fund SeedCamp. Explain how it works will be difficult, because on the website just yet, you can leave your email to receive an invitation. The creators, among which there are, apparently, our compatriot Katia Shutova 28 years, will present their project in the following way: We create a search engine that generates metaphors. Modern search engines return results that are close to your request, which may be useful if you are looking for what the world already knows. But if you are on the border of knowledge is useless. Then metaphors become really need it. says J. Paul Neely (J Paul Neeley), ...

Database replication PostgreSQL-based SymmetricDS

In this article I will tell how to configure database replication for PostgreSQL. For the experiments we use a distribution of Linux CentOS 5.3, although this is not essential. we will use a version of PostgreSQL 8.4.7 and SymmetricDS-2.2.2. What is replication? In fact, it is a mechanism to automatically synchronize the contents of databases running on different servers. As a result of replication of these databases contain absolutely identical data. This is necessary for example in order to provide fault tolerance system (in the case of the fall of the first database server, the work enters the second), or to implement load balancing, different customers can serve different server. For replication you need at least two database servers, so prepare two identical servers with a PostgreSQL database on each. The first is the IP address 10.0.2.20, the second — 10.0.2.21, both gateway 10.0.2.2. You can do a virtual machine like VirtualBox to create two virtual servers and run...

Microsoft introduced the "social" version of Bing

Изображение
Microsoft tries to make its search engine popular, and this, I must say, she manages. In any case, the market share of search is growing, although not exponentially. In order to attract more users, the Corporation adds a search service with new features. This time it was decided to add search to Facebook. As the social network — a huge number of users, the search engine from Microsoft can attract some of these users, taking them from the other search engines (read Google). When you search for a specific request, Bing now shows in a separate column positions of a user's friends in Facebook that are relevant to the query (of course, that shown and the accounts of these friends). It is unclear how such a function may be popular, but developers are pinning great hopes on it. By the way, not so long ago, Google complained that Facebook is a closed sector Network, in which a huge amount of information, and this information is only in this closed world. In Bing, in add...

Extreme testing streaming replication in PostgreSQL 9.1

Изображение
had the task of implementing streaming replication Postgresql 9.1 in production. And since before her business had decided to spend her "extreme testing". This was running two virtual machines VMWare Player with a Ubuntu Server. Installed and configured streaming replication on Postgresql 9.1. sync Speed The host machine was running a script that adds the test table of the replicated database in a loop many rows. The script was running in 3 threads. Virtual machines were created, the load on the CPU (infinite loop in bash) and read the data from the test table. Checked and unbalanced load — it is disconnected on slave, then master. The results turned out very nice picture. Although they were estimated "by eye", but was not seen out of sync for 1-2 seconds. And, this desynchronization occurred only at very high load (multiple concurrent load processes) and for a short time. Stability during power interruptions fsync=off To accelerate upd...