WEBLOG WEBLOG – My IT experience

Docker inspect list ports, volumes and etc.

The `docker inspect` returns useful information about Docker containers.

To filter the returned input one can request a return format.

Main documentation can be found here:
https://docs.docker.com/engine/reference/commandline/inspect/

A peace of information that is not immediately visible, but is always of interest is the list of volumes bonded, which can be extracted as follows:

docker inspect --format='{{json .HostConfig.Binds}}' container_name

To retrieve a list of the binds separated by a new line:

docker inspect --format='{{json .HostConfig.Binds}}' container_name | \
sed 's/"//g;s/\[//g;s/\]//g'| \
tr ',' '\n' 

For the network settings:

docker inspect --format='{{json .NetworkSettings.Ports}}' container_name

To find the command which was used when the docker was started:

docker inspect  -f "{{.Name}} {{.Config.Cmd}}" container_name

To find the environment variables with which the container was started:

docker inspect --format "{{.Config.Env}}" container_name

Many more useful inspect format magics could be found here:

Docker Inspect Template Magic

HTCondor Docker universe throws core.STARTER

This is a problem observed when using HTCondor in the Docker universe.

After re configuring HTCondor and Docker on one processing node, every time a job is sent the following errors are dumped in the corresponding slot’s StarterLog.slot1_N:

(pid:24877) Found 33 entries in docker image cache.
Stack dump for process 24877 at timestamp 1497439919 (13 frames)
/lib64/libcondor_utils_8_7_1.so(dprintf_dump_stack+0x72)[0x7fbbc3e6f0b2]
/lib64/libcondor_utils_8_7_1.so(_Z18linux_sig_coredumpi+0x24)[0x7fbbc3ffb534]
/lib64/libpthread.so.0(+0xf370)[0x7fbbc25bc370]
/lib64/libstdc++.so.6(_ZNSt8__detail15_List_node_base9_M_unhookEv+0x7)[0x7fbbc2f76077]
/lib64/libcondor_utils_8_7_1.so(_ZN9DockerAPI3runERN14compat_classad7ClassAdES2_RKSsS4_S4_RK7ArgListRK3EnvS4_St4listISsSaISsEERiPiR11CondorError+0x42e)[0x7fbbc3e31cee]
condor_starter(_ZN10DockerProc8StartJobEv+0xb66)[0x454656]
condor_starter(_ZN8CStarter8SpawnJobEv+0xc3)[0x45b753]
condor_starter(_ZN8CStarter14SpawnPreScriptEv+0x197)[0x459757]
/lib64/libcondor_utils_8_7_1.so(_ZN12TimerManager7TimeoutEPiPd+0x182)[0x7fbbc3ff9952]
/lib64/libcondor_utils_8_7_1.so(_ZN10DaemonCore6DriverEv+0x9cb)[0x7fbbc3fdb59b]
/lib64/libcondor_utils_8_7_1.so(_Z7dc_mainiPPc+0x13e8)[0x7fbbc3ffebe8]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x7fbbc220db35]

There is also a core.STARTER generated and the output of `gdb /var/log/condor/core.STARTER <<< “where”` is:

Core was generated by `condor_starter -f -a slot1_1 fqdn.domain.com’.

(gdb) Python Exception <class ‘gdb.MemoryError’> Cannot access memory at address 0xb1340bc0:

The lead to that was a bug in the Docker thinpool storage driver, which led to the use of overlay2 driver alongside with a Docker reinstall.

Solution:
There are ‘hidden’ dot files in the condor log directory, they contain cache information that might mess up with you job submission, to fix that one needs to stop condor, remove those files and start condor again. Once done the node start accepting Docker Universe jobs again.

systemctl stop condor
cp /var/log/condor/.s* /tmp/
rm -f /var/log/condor/.s*
systemctl start condor

Eurostat CSV to JSON

Eurostat is great source for open statistical data. Open access to such resource provides students, teachers and ordinary people to analyse and visualise different datasets thus provida arguments and sometimes arrive to fundamental conclusions.

Among the data formats provided from Eurostat are CSV, XML, TSV, HTML, PDF and others. There is also JSON REST API. More information about the API is available here: http://ec.europa.eu/eurostat/web/json-and-unicode-web-services/getting-started/rest-request however this API is not very well documented and it might be difficult to specify the correct filter to extract a given dataset.

The number of requests for graphing and mapping different datasets led to the development of this simple interface which converts a filtered CSV extract of a dataset into a JSON object.

Using the form below you can convert single CSV into JSON:




Joomla WordPress eval base64_encode hack fixes

How did I encounter it eval base64_encode hack

I got to know this hack after I was contacted by my hosting provider telling me that one of my WordPress/Joomla websites was generating a lot of spam traffic. If you are running your own host you might discover it by monitoring the e-mail traffic generated by your server or if the load on the server has unnaturally increased.

The issue spread quite quickly and soon many others websites got hacked. I also got a lot inquiries from friends having the same issue.

How does it happen?

Sites are hacked after an attacker sends through request a piece of code that gets executed once received. That is usually done through untrusted plugins, or flaws in the CMS software.

The attack usually sneaks a line of bad code in some of the CMS’s original files. Usually in the beginning of the files, could also be in the middle. In general they try to replace a line starting with <?php with a line that has many blank spaces followed by something that usually looks like this:

eval(base64_encode('Zwnkjlhfk....'))
Read More

Apache2 LocationMatch with simple regular expression

How to Proxy an URL of the this type:

http://example.com/folder/username/api/some-chars

to an internal server?

I have looked for a long time in the apache2 LocationMatch official docs hoping to find a solution how to write correctly the regular expressions that will do the job until I managed to come out with a working solution.

Read More

Categories