Сообщения

Сообщения за ноябрь, 2017

Backup and restore in PostgreSQL

Изображение
Suppose we have a postgresql in streaming replication mode. master server and hot-standby ready to replace their fallen comrades. When bad scenario, we just have to create a trigger file and switch our app to work with the new master. However, there may be times when legitimate changes were made crooked written a migration and ran to the master and slave server. For example, have been removed/changed data in terms of tables or the tables were not removed. From the point of view of the database everything is fine, but from a business point of view is a disaster. In this case, the proclamation of the hot hot-standby master, the procedure is clearly useless... For warning such a situation there are at least two options... the the to use a periodic backup with pg_dump funds; the to use a backup base backups and WAL archives. The first method is simple to implement and requires minimum effort for installation and maintenance. Put "pg_dump | lbzip2" in cron, and

Optimizing queries. The basics of EXPLAIN in PostgreSQL

Изображение
Why the query runs so long? Why not use the indexes? Probably all heard of EXPLAIN in PostgreSQL. But not so much those who understand how to use it. Himself for a long time could not find a easy to understand tutorial (badly searched?). I hope this article will help those wishing to understand this wonderful tool. This is not a translation, but rather the author's processing of materials Understanding EXPLAIN from Guillaume Lelarge. Some of the information is omitted, so I strongly recommend to read the original. Not so bad Query optimization is very important to understand the logic of the kernel of PostgreSQL. I will try to explain. It is actually not so difficult. EXPLAIN displays the information that is essential to understanding what makes the core of every specific request. We will consider the EXPLAIN command, parallel to understanding what is happening inside PostgreSQL. Described applies to PostgreSQL 9.2 and above. Our objectives: the the to

Stored function With PostgreSQL

Изображение
Hello, habrachelovek! Many of You faced with making the business logic in the DBMS as stored functions/procedures, facilitating client. This is as advantages and disadvantages. Today I'd like to tell You how to create stored functions in the PostgreSQL , written in C. this article the basics that you need to know to start working with them. custom functions At the moment, PostgreSQL allows you to define custom functions the following types: the SQL -functions the feature C the functions for the procedural languages ( plpgsql , pltcl , plperl , etc) SQL -functions are functions, in the body which have one or more SQL requests, while vozneseny the result will be the result of the last query. If vozraschenie the result is not void allowed to use INSERT , UPDATE , or DELETE RETURNING . Functions in C are statically and dynamically loadable. Statically loaded (also called internal functions) are created on the server when initializing the database cluste

Holland — backup of MySQL/PostgreSQL without headache

Изображение
one of these days I'm tired to use samopisnye scripts to create database backups. No matter, they were developed by me or found somewhere on the Internet. Based on the principle that time is the most expensive resource of the system administrator (engineer, architect), a solution was found that meets the following requirements: easy installation, quick setup, and, as the sum of the previous requirements, fast operation. According to official website , Holland — freymvork open source for creation of backup copies, designed by Rackspace and written in Python. The project aims to create backups with great flexibility, logical structure and ease of use. At the moment Holland works with MySQL and PostgreSQL, but in the future will include a greater variety databases, and even applications not related to databases. Due to the modular structure of Holland can be used to create backup copies of anything as you like. Let us imagine that our scenario is a daily backup of one MySQL

Russian sound: my attempt of production and crowdfunding

Изображение
How realistic is a Russian citizen on one salary to start producing electronics? Sounds crazy, but I decided for fun to try to make a trial batch of Russian sound cards and raise money using crowdfunding. Of course breaking a psychological barrier — the belief that in Russia to launch high-tech production is impossible. In the ultra-modern and innovative Russia for most of "Russian" equipment producing countries in South-East Asia. I also wanted to try to make something of their own with Russian manufacturers. To do this we have a very expensive, because all the "civilian" technologies have long been abandoned. But if the process to join the other passionate people, we can come to the conclusion that to engage in the production, we will have no less favorably than repurchase. In the end, the producing, the entrepreneur receives a higher quality sense of self, national pride, or something. device Selection Criteria for device selection were simp

Reset PHP cache through SQL query or from a gun on sparrows

Изображение
Good time of day. Faced with such a task using an SQL query to reset PHP cache. Simply put, to remove multiple files in a specific directory. At the entrance there is: the the DBMS PostgreSQL 9.3 the OS – Linux (Fedora, but not an essence important) In the end, it turned out the decision to bundle python + C + bash (just a little). A little Unix-way, but maybe useful to someone. Wrap the command to delete the PHP cache in a bash script, thinking about his further expansion of new functionality. Generally speaking, it would be nice to pass the desired command as a parameter. Here is the script: the #!/bin/bash function __clear_cache() { rm /var/www/html/cache/*.php } FUNCS=() FUNCS+=("__clear_cache") function function_exists() { local e for e in "${@:2}"; do [[ "$e" == "$1" ]] && return 0; done return 1 } if function_exists "$1" "${FUNCS[@]}"; then eval "$1" else echo "Function $

The most popular article in a scientific journal

Изображение
At the beginning of 2000-ies the scientific journals only began to publish their materials online. Most readers were drawn up for a paid subscription for the paper version, and purchase the digital copies were rare. However, publishers understand that the future is in the digital delivery of content, so opened sites and closely monitored the attendance recalls molecular biologist Richard Grant in his blog. One of the first online version was opened by ("Journal of Molecular Biology") from Elsevier. As expected, at first the attendance was very small. But soon the editors noticed that one of the articles in issue 324, vol. 1, 15 November 2002, enjoys high popularity. Attendance this page was many times more than all the rest. The article was entitled "Molecular analysis of the tissue localization and relationships of the Ca 2+ ryanodine receptor of Caenorhabditis elegans". Guide magazine asked the question, what is so interesting in a scientific articl

JSONB in PostgreSQL queries

Earlier I wrote how to enable jsonb postgres/psycopg2 . Today experimented with how to query data in columns of type JSON. On this subject there is documentation but I was not quite clear how to operate different types of transaction: the CREATE TABLE json_test ( id serial primary key, data jsonb ); INSERT INTO json_test (data) VALUES ('{}'), ('{"a": 1}'), ('{"a": 2, "b": ["c", "d"]}'), ('{"a": 1, "b": {"c": "d", "e": true}}'), ('{"b": 2}'); The query worked, let's get all the data to check: the SELECT * FROM json_test; id | data ----+-------------------------------------- 1 | {} 2 | {"a": 1} 3 | {"a": 2, "b": ["c", "d"]} 4 | {"a": 1, "b": {"c": "d", "e": true}} 5 | {"b": 2} (5 rows) Now make the filte

Factory sites (F. CMS) lost the court

I'm Sure many active MODX developers heard about the "Factory sites". These are the guys that made the copy engine of MODX Revolution, called it F. CMS and rivet on it the sites, posing for your own engine. But besides the fact that they took a third-party opensource engine, they also "borrowed" various third-party modules, just renamed them and sell as their own, not without any authorship. Among the victims — andchir and bezumkin . Last wrote about this situation pretty long and detailed. Last year I had to face a couple of clients whose websites are long and hopelessly was developed in the Factory. One of the sites was redesigned and the results were written a detailed topic , where I judged the quality of the work performed by said company. And Yes, I transferred the website was developed by F. CMS is really very much like MODX Revolution, and Yes, as a module of the online store used by ShopKeeper Andchir . In fact, a continuation written in my ar

Amazon failed diesel generators

Изображение
Amazon explained why the blackout of the data center in the region of East-1 in Northern Virginia, bringing the June 30 gone offline Instagram, Netflix, Pinterest and other sites. The reason he called incorrect operation of diesel generators, specifically, raspredelitelno flap (switchboard), which was to switch the system from one channel to another. After the jump voltage "generators start normally", but if one data center East-1 powered from generators without incident, the second data center, "each generator separately, could not provide stable voltage after you run them," explains Amazon. As a result, the network is not automatically switched to the power from the generators, and UPS'Ah soon ran out of charge. Diesel generators were installed in the data center in late 2010 and early 2011, prior to the installation they were tested by the manufacturer. After installation in the data center they again drove through eight hours of tests, ev

How many servers in the cloud Amazon EC2?

Изображение
Like many other Internet companies, Amazon doesn't disclose information about their infrastructure, including on the number of servers. But analyst from Accenture Technology Labs has made its own calculation: it happened that seven data centers of Amazon Web Services should be at least 454 400 servers . Interestingly, 70% of them are located in region US East (Virginia). the number of server racks number of blade servers the US East (Virginia) 5030 321 920 the US West (Oregon) 41 2624 the US West (Northern. CA) 630 40 320 the EU West (Ireland) 814 52 096 the AP Northeast (Japan) 314 20 096 the AP Southeast (Singapore) 246 15 744 the SA East (Sao Paulo) 25 1600 the Total 7100 454 400 Amazon uses a regular pattern of assignment of internal IP addresses: each instance gets the address of the 10.x.x.x, and each rack gets a range of 10.x.x.x/22, it is possible to identify all the virt

Integrate Amazon S3 .Net app

Изображение
Recently I faced the task of use in your application to the cloud storage. Not stalkivalis this before, I thought that this task will take more time. But, to my surprise, already developed enough convenient tools for interaction. An example of their use is described in this topic. Consider how files are stored on Amazon: we have Basket (Buckets) directly in which you can upload files and create directories. Access is through two key Access Key and Secret Key , which you can view on a separate page of the profile. / > So, we have a task to implement in my application the ability to upload files and delete already downloaded. Create a new Windows Forms project. Add using NuGet Amazon SDK the PM> Install-Package AWSSDK Let's add on our form two ListBox'a: one to display the list of baskets, the second to display the list of files in the selected basket. Also, add a button to get a list and upload the file to the specified bin and delete the selected file:

Speeding cars — project for motorists

Изображение
There is one good principle for a person who wants something to take, and do — so much so that his creation became interested in the General public: do only what will benefit itself. The desire to realize some project for Android came to me ten minutes later after gaining smartphone with this OS. But I have always respected this principle, therefore, the ideas passed rigorous selection process and were weeded out as unnecessary. In addition to the smartphone, I have another car, and he appeared much earlier. You can safely put me in motorists. Several times I had a desire to measure real-time acceleration to 100 km / h his car. It was not strong enough to buy additional devices to understand — and I just took the stopwatch, looked at the speedometer and got some figure. However, the speedometer was lying, from different angles look different, the arrow didn't crawl too fast to clearly define the moment of reaching the desired mark. Having caught the phone, I started to search

Cloud-based protection .NET applications

Изображение
Perhaps protection has always been one of my most favorite topics. I loved to come up with a complicated clever examination of license program, and enthusiastically implement them. I've always held to the principle that a hacker to break in, needs to study the high technology used in the program. Let him think about thread synchronization, if he wanted to put a breakpoint in the validation algorithm of the key. Let the studying questions of the COM reference count, if he wants to intervene in my algorithm. Let him think about the representation of bitmaps images in memory, if he decided to figure out how I kept the key. Yes, C++ was almost an ideal language in this respect. But times change, old technology gone and in their place come new, more productive and comfortable. So our team moved on .NET. But in exchange for ease of development and ease of debugging, we in addition got in the appendage and ease of decompilation. The attacker could not simply bypass the license c