PyPCP and PCP CLI for PgPool-II

PyPCP interaction with PgPool-II

Pgpool-II Communication Protocol (PCP) library and pcp_* utilities which are used to manage & monitor Pgpool-II cluster. Pgpool-II and all the utilities are Linux based. To develop application and web interface you have to be on some *nix system, which limit the choice of integration of Pgpool-II with other systems and platforms.

PCP Library

To overcome this limitation, I developed a Native Python based PCP library (which does not require Pgpool-II/PCP library) with name pyPCP. Being written in Python language it can be used on any system. pyPCP provides an interface for administrators to perform management operation, such as getting Pgpool-II status or terminating Pgpool-II processes remotely. You can use library get statistics about Pgpool-II interaction with client application and backend PostgreSQL servers. Pgpool-II installation is not required on local machine. Currently It is tested on Mac, Linux and Windows systems.

How to Install

You can install and update pypcp using pip.

$ pip install pypcp

Example Usage

from pypcp import PCP, ResultStateType

pcp = PCP()
pcp.pcp_connect('remote_ip', 9898, 'postgres', 'secret')
result = pcp.pcp_node_count()
if result and pcp.PCPResultStatus(result) == ResultStateType.COMMAND_OK:
     print('Node Count  : ', result.pcp_get_data(0))
pcp.pcp_disconnect()

Result

$ Node Count  : 2

JSON support

You can also get results for each command in JSON format.

print(result.pcp_get_json_data(0))

Result:

$ {'command_status': 'success', 'node_count': 2}

PCP CLI

At the moment there around 13 pcp_* utilities are being built along with Pgpool-II. By the time we add more features/commands in PCP, number of these utilities will increase.

PCP CLI combines functionality of all those utilities in a single utility. Same as Azure, Github CLIs, pcp_cli provide groups and sub-groups of command based on relevancy.

Following example show ‘pcp_cli node count’:

$ pcp_cli node count -U postgres -H pgpool_host -v
Password:
Node Count
____________
2

Following is the list of pcp_cli commands & sub-commands:

pcp_cli pool status
pcp_cli pool stop
pcp_cli pool reload-config
pcp_cli pool check-health-stats
pcp_cli node count
pcp_cli node info
pcp_cli node attach
pcp_cli node detach
pcp_cli node promote
pcp_cli node recovery
pcp_cli process count
pcp_cli process info
pcp_cli watchdog info

Conclusion

PyPCP and PCP_CLI will provide users freedom to develop applications and website across platforms for management and monitoring PgPool-II cluster.

PyPCP is open source and can be found at following github repository.

https://github.com/uhayat/pypcp

PgPool-II CRL Support

pgpool_ssl_crl

The Certificate Revocation List (CRL) contains blacklisted entries by a Certification Authority (CA). CA can revoke certificate prior to expiration date because of different reasons as mentioned in RFC 5280:

  • unspecified
  • keyCompromise
  • cACompromise
  • affiliationChanged
  • superseded
  • cessationOfOperation
  • certificateHold
  • removeFromCRL
  • privilegeWithdrawn
  • aACompromise

PostgreSQL already support CRL via its ssl configuration. On the other hand in PgPool-II till 4.1, SSL implementation for authentication between frontend-to-pgpool lack this functionality. To cover this short coming, a feature is committed to PgPool-II 4.2 branch:

commit: dc3116c797aab862635f2f7c73c6cbd35cd167cc
author: Tatsuo Ishii <ishii@sraoss.co.jp>
date: Sat, 14 Mar 2020 07:54:38 +0500 (11:54 +0900)
Add support for SSL CRL (Certificate Revocation List).

Patch created by Umar Hayat, along with Japanese document by Tatsuo
Ishii.
Discussion: https://www.pgpool.net/pipermail/pgpool-hackers/2020-February/003505.html

This commit introduced a new configuration variable in pgpool.conf file.

  • ssl_crl_file , to specify path to crl file

By default this configuration variable is disable in pgpool.conf file. If you specify CRL file path using  ssl_crl_file  configuration variable, it will be loaded at startup.

How to generate CRL file:

When you create a certificate, lets say with CN=postgresql, You will see an entry in openssl database file ( i.e. index.txt file)

V 210328050942Z 1000 unknown /CN=postgresql

After revoking the certificate the same entry in index.txt file, will look like as following:

R 210328050942Z 1000 unknown /CN=postgresql

You will notice that ‘V’ is replaced by ‘R’, which means certificate is in revoked states. To communicate the same to server, you can generate a CRL file using following command:

$ openssl ca -gencrl -out server.crl -cert root.crt -keyfile root.key

If you which to see the contents of CRL file, you can do so using openssl ‘crl’ subcommand. Command will show the certificates serial number along with its revocation date:

$ openssl crl -in server_revoked.crl -noout -text
Certificate Revocation List (CRL):
        Version 2 (0x1)
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: /CN=MyRootCA
        Last Update: Mar 18 05:09:42 2020 GMT
        Next Update: Mar 18 05:09:42 2021 GMT
        CRL extensions:
            X509v3 CRL Number: 
                4097
Revoked Certificates:
    Serial Number: 1000
        Revocation Date: Mar 18 05:09:42 2020 GMT

How to configure CRL in PgPool:

Once you have CRL file created, next step is to configure it with PgPool. You can place it on PgPool server and update the path in pgpool.conf file like bellow:

ssl_crl_file = './server.crl'
                                   # Path to the SSL certificate revocation list file
                                   # (change requires restart)

For CRL, thats all you need to configure, not to mention that CRL will only work if you have already working front-end to PgPool working SSL connection. CRL file is loaded on PgPool start up only. Updating any thing in this configuration would require PgPool restart.

Run PgPool with CRL configuration:

After updated CRL configuration, you can start PgPool. Now if we try connection with revoked certificate using PSQL.

$ psql -h 127.0.0.1 -p 9999
psql: error: could not connect to server: SSL error: sslv3 alert certificate revoked

As you can see, even if certificate is valid, but it revoked due to any reason, client will not be able to authenticate.

 

 

 

National ICT Scholarship Program 2008 FOR A PROSPEROUS PAKISTAN

ICT R&D logo

National ICT R&D Fund is offering fully funded scholarships for talented students from rural/non-metropolitan areas of Pakistan. The National ICT Scholarship Program 2008 (NICTSP 2008) will provide 550 fully funded 4 year undergraduate scholarships in Information and Communication Technology (ICT) related disciplines in the Top ICT universities of Pakistan.

National ICT R&D Fund will also facilitate Training of Teachers and Foundation Training for students to enhance the skills needed for attempting aptitude based university admission tests.

Students will be required to appear for a customized MCQ based aptitude test conducted by National Testing Service (NTS) for National ICT R&D Fund. They will also need to qualify for university admission tests to earn this scholarship. learn more

National ICT Scholarship Program 2008 FOR A PROSPEROUS PAKISTAN

National ICT R&D Fund is offering fully funded scholarships for talented students from rural/non-metropolitan areas of Pakistan. The National ICT Scholarship Program 2008 (NICTSP 2008) will provide 550 fully funded 4 year undergraduate scholarships in Information and Communication Technology (ICT) related disciplines in the Top ICT universities of Pakistan.

National ICT R&D Fund will also facilitate Training of Teachers and Foundation Training for students to enhance the skills needed for attempting aptitude based university admission tests.

Students will be required to appear for a customized MCQ based aptitude test conducted by National Testing Service (NTS) for National ICT R&D Fund. They will also need to qualify for university admission tests to earn this scholarship.

This program will be run with the financial collaboration of Public Sector Development Program (PSDP).
learn more….