Pages

March 12, 2017

Secure Logging Training by Clarified Security


This is my second training in Clarified Security, first one was Web Application Security. This one is shorter – only one day, – and trainer was Mait Peekma.

The most useful part for me was about what and how to log, but unfortunately it was the shortest one. As always, logging depends on the context of your application, so the only thing that public training can give you is the understanding of attacker's logic (like log evasion and tampering) and the sense of importance of logs in software security – both these things were done good.

In summary, good training to start. If you participated at Web Application Security before, then you already know some things, but in Secure Logging they were expanded. Also I had a chance to perform some attacks that I have heard about, but never succeed to do them by my self. I suggest it for those who should deal with incidents or just care about security.

February 11, 2017

Automatic W3C Validation Using YAML Conf in GitLab Continues Integration

As I already wrote I host my Profile Page on GitLab and use Runner to deploy the code. Runner is quite powerful tool, than can be managed with YAML .gitlab-ci.yml conf file.

Basically, it means that after every commit some automatic actions happen (well, actually, not on every commit, depends on how you configure it). In my case .gitlab-ci.yml looks like this:

stages:
  - deploy
  - test

pages:
  stage: deploy
  script:
  - scripts/move-content-to-public.sh
  artifacts:
    paths:
    - public
  only:
  - master

w3c-validation:
  stage: test
  script:
  - scripts/w3c-html-validation.sh
  only:
  - master

It means that after I commit something into master branch it performs deploy stage (script move-content-to-public.sh) and test stage (script w3c-html-validation.sh). Both scripts are my own custom sripts and in this post I want to talk about the second one – W3C validation of HTML file.

You can find the script content in my repo: w3c-html-validation.sh. The good thing is that W3C has public API for validating some URL. So all I need is just requesting this API with URL of my page as an input parameter. And that's the reason why I perform the test stage after the deploy stage: new changes need to be published to check them. Also, W3C errors are not verty critical, so I dond't want to fail the whole deploy because of them.

It looks like a very simple thing (it actually is), but it took a time for me to configure this validation (as I was not very familiar with YAML), so I decided to leave some notes here.

February 9, 2017

Profile Page 2.0

Two weeks ago I published my Profile Page. Since then some changes were made, that I think are worth sharing.

First of all, blog changes. As I have now profile page I don't need to keep same staff here, so blog design has been simplified: pages with extensions and the right panel with some weird staff has been removed to give more space to post content. I think I will make some more design changes in the near future.

Second big staff: both FireFox extensions URL&p and JIRA Issue Opener has been finally refactored and redesigned. So now they are in consistancy with Chrome extensions and should properly work with final FireFox versions.

After reading an article An opinionated guide to writing developer resumes in 2017 I decided to remove skill progress bars from the page and CV. Now there are two sections of Strong and Knowledgeable technical skills. Also short descriptions of previous experience has been added to CV.

One more modern thing that I have completely forgotten about is page sharing in social media. So this time I added some meta tags to help Facebook and other medias better share my page. Good article about sharing: A Guide to Sharing for Webmasters.

At last, I have encoutered interesting fact: fixing general and fundamental architecture things automatically fixes several small bugs. Of course, I knew it before, but I have never experienced it personally. For example, I am a fan of SVG pictures, but in Microsoft Edge they looked strange. I didn't pay much attention to this compatability bug, but at some point I started to optimize pictures for web. And it turned out, that you should never use text as a picture. If you have only text you should use HTML+CSS (and almost always it's possible to get the same result as with picture). So I migrated all headers in profile section from pictures to text and the Edge problem was automatically fixed.

January 29, 2017

Profile Page with HTML5 and CSS3

I decided to learn about new web technologies, so I have a plan to build a few small projects. Basics first - profile page from scratch (without any CMS-s and templates) - irina-ivanova.gitlab.io.

All the source code of the page and description about used technologies can be found in GitLab repo gitlab.com/irina-ivanova/irina-ivanova.gitlab.io.

Shortly: HTML5+CSS3 (and a little bit of AngularJS to show last 3 posts from this blog). New challenges for me were responsive design (which turned to be not so hard as I thought), SEO and performance (images and fonts optimization, SVG, content optimization etc).

Also I tried to use new Git manager GitLab and I love it! GitLab, unlike GitHub, allows to create private repos for free. Also there are a lot of free build in functionality like milestones, issues, Kanban board, continues integration, hosting the page etc. I'm not only host this new profile page there, but I migrated all my extensions and scripts from GitHub. Speaking of extensions, I redesigned them all (actualy, only Chrome ones for now), so they should be more fresh and modern now. GitLab has one downside: UI is a litle bit slow and continues integration sometimes pending the changes for an hour (but not too often).

If you are interested in details about used tehnologies and resources - see README file in the repo. There are some links on articles that being used in the process.

Some surprises during this experience:

P.S. The day I was planned to publish the page and this post I've got a subscription email with the article An opinionated guide to writing developer resumes in 2017. A lot of things in my CV are incorrect and I plan to change it in the near future.

November 19, 2016

Use Oracle Data Base in Command Line


Cadmus Asks the Delphic Oracle Where He Can Find his Sister, Europa by Hendrik Goltzius, 1615


Sometimes I need to make some quick changes in data base. Or I need to do several changes using some patterns and templates. In that case I don't want to open editor (I use DataGrip) and wait for loading all resources, connecting to DB, opening SQL window and inserting or copying SQL query. I am that kind of person who keeps open only those applications that I am going to use in next 30 min. If I don't plan to use something in next half an hour I close it, so every opening takes some time and effort.

The best way to save time and effort is command line or, actually, scripts. SQLPlus is a great tool for using Oracle data base in the console. You can find instructions about installing SQLPlus on MacOS on StackOverflow: Oracle Sqlplus client on Mac.

The annoying thing about using data base in the terminal is connecting: you have to remember and write all credentials of data base every time you want to connect to it. But, as usually, scripting resolves this problem. I wrote simple Bash script connect-to-oracle-db.sh, that opens connection using small alias as an input. For example, I write sql test in terminal and it opens connection with test data base where I can write some SQL right ahead. Instead of test it can be any base that I work with: demo, live, production etc.

Second script that I use is changing some value in specific data base in specific table. Some times one of the clients asks me to change specific data in their DB, so now instead of opening an editor I can just run a script with parameter given by client and it changes all the data, which saves me time and doesn't disturb my attention on other tasks so much. The script looks something like that:

#!/bin/bash

parameter=$1
sqlplus= # Path to SQLPlus
username= # DB username
password= # DB password
db= # DB name

SQL="
  UPDATE some_table
    SET some_column = some_value
    WHERE some_other_column = '$parameter';"

"$sqlplus" "$username/$password @$db" << HERE

$SQL
commit;
quit
HERE

Of course, SQLPlus doesn't replace editors like SQL Developer or DataGrip, but it can save a lot of time and effort in performing small and routine tasks.

October 24, 2016

Why I Don't Like Testing Conferences


The painting: "The Witches’ Cove" by Jan Mandijn

The best book about software testing has following introduction: "This book is about software development as we've experienced it." ("Lessons Learned in Software Testing" by Cem Kaner, James Bach, Bret Pettichord). Because you can't talk about testing without the context of the general development itself.


I like conferences – they are usually very inspiring, motivating and sometimes challenging. Visiting testing conferences gives a lot of ideas how to do my job better, but almost always that means improving some processes at the project. And it's almost impossible to change some steady process if more than 10 (or even 5) people are involved in it. To change the process you need to convince all team members that it brings benefits to the project or product. And to convince team members you need to retell the story you heard on the conference (which usually is as long as the conference talk or even longer with all the preparations you need to do) and to be talented speaker (usually inspiring speakers at the conference are good at speach), which in major cases is not true. So, wouldn't it be better that all (or maybe the key ones) team members just visited the conference all together to hear the same talk from experienced speaker and be inspired all at a time?

Majority of the people in the audience is having some ideas during the talk, they are very inspired and willing to do some changes in their project, they came back to work and start to talk about these changes with developers or project managers and then... they get couple of arguments why it won't work in this specific project. And this person, who visited the conference, is not so skillfull as the speaker to pitch other team members. So everyone thinks he is a boring tester who constantly offers some silly ideas.

This is not just impractical, this is harmfull. It brings discord between programmers and testers (and analytics, but programmers vs tester is the most popular confrontation). Programmers doesn't understand why testers suggest to do their life harder, because they haven't heard the same speach. For example, the idea to involve testers into the development as early as possible may seem to be silly if you hear that from one junior tester who visited some conference ("he doesn't understand anything at the early stage and I don't have time to explain it" – may say some programmer). But the same idea from the experienced speaker on the stage is not so silly anymore (at least you need to have a proper argument to argue with it).

I'd like to have conferences about software development generally, where all roles can participate. Surely, there should be specific conferences for testers and programmers, where speakers may talk about how to automate tests, which tools can be used, how to do security or performance testing. However questions like why we need automation, why we need security, at what stage of the project we need to thing about security, how ofter we need to release updates in production should be convered in general conferences, because these are the problems where all team members are involved. The problem is that I don't know any widely spread good conference about software development generally - all good conferences are role specific.

After all, all team members have one common goal – to create a product (good teams have goal to create a qualitative product). Both testers and developers works for the same goal, but visiting different conferences they start to see the same goal from two different perspectives.

September 7, 2016

Checking Deployments on Tomcat Server Without Web Manager

In some cases Tomcat web manager is disabled (for example, in production for security reasons). Then the only way to see deployments and their statuses is to use Tomcat API. To list deployed applications you may do following request:
http://localhost:8080/manager/text/list
And you will get something like this:
The problem is that if you have a lot of applications such output is not very easy to read – you can't say is there any stopped applications without reading all rows. For that case you can use awk to color the output, which is much more informative:
The next problem is that awk command is quite long and you don't want to type something like this every time:
curl http://localhost:8080/manager/text/list | sort | grep ^/ | awk '{ gsub("running", "\033[32m&\033[0m"); gsub("stopped", "\033[31m&\033[0m"); gsub("\\:[0-9]+", "\033[34m&\033[0m"); gsub("^/.+:", "\033[36m&\033[0m"); gsub("[0-9]+$", "\033[33m&\033[0m"); print }'
So you can use a script, that returns you colorful list of deployments without typing any URLs and regexps:
./show-status-of-applications-tomcat.sh

There is one other case, when script is even more convinient than web manager – if your environment is running on many servers and clusters (like production, again). In that case script can show information about all clusters and servers on one page:
I have separate script show-status-of-prod-applications-tomcat.sh for that, but it's possible to merge them into one file or modify it to work with parameters (for example, give a manager URL as a parameter).

All scripts are available in my GitLab repo gitlab.com/irina-ivanova/scripts and don't forget about aliases!

August 9, 2016

Probe Testing in The Martian

The story from The Martian about testing the probe is worth a post (some scenes and phrases are skiped).








July 8, 2016

Bachelor Thesis "Version Update Automation Using Scripting Language Bash"


I've finally finished University of Tartu, so I am a Bachelor of Science in Engineering now!

My work was about version updater bash script, that I made on my work for updating Java applications on different web servers. I once wrote a post about first version of this script – Scripting For Automated Update (Tomcat 6) [DEPRECATED], – the final version has more features and much more code.

The thesis is written in Estonian and can be found in GitHub – Version Update Automation Using Scripting Language Bash.pdf (or in UT registry).

The script itself is open-source (currently only Tomcat 8 part) and can be used in other projects. It's located in GitHub repo – github.com/iriiiina/version-updater, the manual about install, configuration and usage is in GitBook – iriiiina.gitbooks.io/version-updater-manual.

And here are slides of defence (also in Estonian). I don't know why you may need them, but since we are talking about thesis I'll leave them here:

I feel proud that I got A for the thesis, but I don't have any good feeleings about finishing the university itself. Two years ago I wrote a post Secrets of a Buccaneer-Schoolar by James Marcus Bach where I explained my opinion about the university and hight schools, so it wasn't the priority for me. I decided to finish just to not loose already gotten points for finished courses – I've already passed 99% of the programm, so it would be a shame to spend that time for nothing.

June 17, 2016

Bash Scripts for Transfering Files Between Server and Local Computer


Yes, Bash scripts again! Two scripts for copying files from server to local computer and vice versa. Usually I use them if I want to work with some text files in graphical editors, not in Vim. So I can download the file to my local computer and then upload it back to the server.

./copy-file-from-server.sh [SERVER] [FILE]
./copy-file-to-server.sh [SERVER] [FILE]

Basically, it's the same as using scp command, but scripts allow you to define short names for servers. For example, instead of username@192.168.1.1, username@some.hostname.com or something1-23-45-678-9.eu-east-0.compute.amazonaws.com you can use short names, like test, demo, latest etc.

To set your own hosts in the script you need to modify function setHost(). In the copy-file-from-server.sh script you can also set a fixed directory where you want to download files. Currently it downloads to the same directory where script was run, but you can change it in the downloadFile() function.

Also, don't forget about aliases! You can set some short name to the script and use it in every directory.