Curl net performance test

Print out CloudFront X-Amz-Cf-Id when response time is slower than set threshold.

#!/bin/bash

output=($(curl -I -s -w "Time: %{time_total}\n"  http://cf_url.here grep -e X-Amz-Cf-Id -e Time | awk {'print $2'}))

id=${output[0]}
time=${output[1]}
compare=0.01

if (( $(echo "$time > $compare" |bc -l) )); then
    echo $time - $id >> test_results.txt
fi

Small variation (unrelated to CF)

#!/bin/bash
while :
do
output=($(curl -o /dev/null -s -w "time_namelookup: %{time_namelookup}\n time_connect: %{time_connect}\n time_appconnect: %{time_appconnect}\n time_pretransfer: %{time_pretransfer}\n time_redirect: %{time_redirect}\n time_starttransfer: %{time_starttransfer}\n time_total: %{time_total}\n" http://services.realestate.com.au/services/listings/120542701 | grep time_ | awk {'print $2'}))

time_namelookup=${output[0]}
time_connect=${output[1]}
time_appconnect=${output[2]}
time_pretransfer=${output[3]}
time_redirect=${output[4]}
time_starttransfer=${output[5]}
time_total=${output[6]}
compare=0.1

if (( $(echo "$time_total > $compare" |bc -l) )); then
echo $time_namelookup,$time_connect,$time_appconnect,$time_pretransfer,$time_redirect,$time_starttransfer,$time_total >> test_results.txt
fi
sleep 1
done

Replace Mercedes-Benz B250 cabin(pollen) filter replacement DIY

Quoted $300+ from dealer shop which I know a serious rip-off price for mere carbon filter.

Searched eBay, local stock comes with some $100 price tag, still too steep.

Found a German seller, selling compatible filter for approx. $27 for the item and $15 shipping – even made in Germany!

The confusing part was finding b250 cabin filter replacement in YouTube resulting the previous model, W245(2005-2011).
Any 2012~ model need to search for W246 instead, like this.

5-10 minutes work, don’t even need a screw driver. Boom, saved $250+.

MongoDB to DynamoDB

Attempted to convert rather large mongo dump to dynamo.

Seems like AWS data pipeline is most elegant way to do this if one can workaround with dynamo non-standard json format issue.

Get https://www.npmjs.com/package/dyngodb2 and create a collection

export json and run some sed to trim

mongoexport --db=wiki --collection=wiki -f title,text > output.json
sed -i 's/{ \"$oid\" : \"/"/g;s/\" }, \"/", "/g;s/{{{//g;s/}}}//g;s/{{//g;s/}}//g;s/$/,/g;' output.json

split file to avoid OOM, adjust 30000 based on your free ram then add [ and replace trailing, with ]

split -l 30000 output.json
sed -i '1s/^/[/;$s/,$/]/' x*

import & make a batch process to do all sliced files

dyngodb2 < db.wiki.save(json('xaa'))
{
  "operation": "echo",
  "TableName": "Thread",
  "Key": {
      "title-index": "!"
        }
}

Europe Travel 2016

Turkey – Istanbul
Good: Nice people and food
Bad: Syrian refugees(=security risks), bit too excessive street sales

Turkey is way more developed country than I thought therefore booking cheap hotel was a bit of mistake. It was too cheap as burger king meal is AUD 10 ish hence rather crappy, even housekeeping didn’t given at all time. Also its right next to a Mosque hence 6am morning pray coming from huge speaker was the morning call. Public transport here is quite good so suggest invest bit more money and get somewhere outer city location.

People are nice apart from excessive street marketing folks. Maybe little bit too nice as I can see bunch of Syrian refugee on the street which is sympathetic however a security risk as well. Yes, there was car bombing on military base in Turkey capital even during our stay. Its a good place in general. Tourist rip-off isn’t too bad tho buying from spice market instead grand baza is a better idea.


Greece – Santorini
Good: Nice view
Bad: Need to go down all the way via walk for nice view, things are quite expensive as a holiday destination

Most accommodation is on cliff side for view hence quite far from where you can get by car. So I had to carry all the┬áluggages like a KM then the reception lady mentioned they do have porter service. :S Its mainly relaxing place and everything is overpriced as the island economy itself relies on tourist. e.g) local wine is absolutely poor – Aussie wine heightened my wine standard.- People speeding in general on remarkably curvy and hilly roads and parking like badass. I, too enjoyed park anywhere mood and ends up parking in front of port authority office in day 2. ­čśŤ It wasn’t too bad driving experience as it was off-peak season but probably much worse during the peak season.


Greece – Athenes
Good: Acropolis
Bad: Nothing worth to see apart from Acropolis

about the same as Santorini but with better public transport access. Still people smoke everywhere incl. Indoors. Major attraction is acropolis and other ancient cities which is quite a view. Tho need to pay every single this and that site. There’s bundle ticket that could save some bucks. Its all within walking distance so can travel without too much hassle even without public transport access. Internet is slow for both mobile and wifi, seems like greek telcos are not affordable to increase bandwidth.


Hungary – Budapest
Good: Cheap Thai massage and Thai food
Bad: “We had this Nazzi issue then Soviet” – kinda gloomy area

Oddly the best thing you can try here is (relatively) cheap Thai food and massage. Its about half or less price compare to Australia. Not only Hungary but all not so wealthy European country in general, the preservation and restoration status of historic site is rather poor as it costs too much money. Hence isn’t really impressive thing to see. Esp. When you saw western Europe palace etc first then came to see eastern one, it will be much disappointing.


Poland – Krakow
Good: Pubs, food, dragon born(!) etc
Bad: “We had this Nazzi issue then Soviet” – kinda gloomy area
Place where you can travel to notorious Auschwitz. Food is cheap and in good quality. Funny the city is named after dragon slayer and dragon born is displayed in front of city museum. Also local currency is zolten which means gold in polish.


Czech – Prague
Good: Can’t seem to find any Good?
Bad: Historic site restoration and preservation status is horrible
Cant really think of anything good out of Prague. Way too overcrowded and as usual, historic sites are not really well preserved.


Czech – Cesky Krumlov
Good: Nice little town
Bad: You can travel all the places within half a day, too small
Nice little town and souvenir is much cheaper than Prague and has good quality too.


Germany – Berlin
Good: Can visit German parliament for free, WW2 history lesson, nice people
Bad: Sausage isn’t great as expected. City is becoming a giant slum.
I can sense it ‘was’ a good city. Although likely due to recent refugee intake, the city appears to be experiencing some security issues. From hotel to train station, there’s drug dealer every 10m as if they have some kind of invisible booth between them. Dozens of homeless having party every night in front of supermarket and the owner seems can’t really take any actions.


Italy – Rome, Venice and Florence
Good: FOOOOOOOOOOOD
Bad: Customer service is close to NIL

If you see nice customer service and those people hand out English menu on the street, I can say they’re all tourist trap. The real nice places are usually in some back street alley and has filthy customer service. Speaking English upfront is particularly not a good idea in this area. I suppose you just need to try a bit, technology assistance widely accepted hence Google translator is your friend.

TIPS
If you check diner and see a priest dining, that’s your diner.
Good gelato place never visibly display the product.
Buy your water from supermarket in advance all the time, 1000% mark up price is everywhere otherwise.
Internet booking is your friend. Ticketing queue is seriously long for everywhere.
There’s some leniency for Internet booking so don’t panic if you’re running late for a bit.


Switzerland – Zurich, Bern, Interlaken and Lucerne
Good: “I want to buy this” level of product everywhere
Bad: Racism

In Zurich and Bern, I’ve seen three Toyota during my entire stay. Appears to be people have so much money to burn, heard double average income compare to entire Europe. Also unlike typical expectation of swiss alps etc, has all sort of luxury goods store.
Swiss local made goods aren’t cheap either, some little watch with 4 digits price tag is quite usual.
Jungfrau was good but unsure if it worth approx. $150/pp ticket.
All Swiss highway has 120km/h speed limit and people drive like Madmax for snowy, hilly and curvy road.
Racism goes really high here, can see people hate Asian from guts. It might be somehow logical as over 50% of crime committed by foreign nation here but Asian for crime hmm.
People are generally quite sloppy, had more than a few issues during our short stay here. Ensure double check everything. e.g) bookings, orders, amount of change etc

Austria – Vienna and Salzburg
Good: Even busker plays remarkably good music here
Bad: Racism

Suggest book any concert and/or show every night. This is what makes this city worth. There’s some other things such as palace etc but you will probably see more than enough palace and cathedral here and there.

About the same Racism issues as Swiss. Locals are tend to close eyes for such act.


France – Paris
Good: …everything?
Bad: Level of security isn’t really great even after a few incident.

I seriously couldn’t find anything bad about France, or least for Paris. I’m really impressed even one of most traditional show, Moulin Rouge, has different races and least one Lesbian dancer.
People are also nice and have certain confident like “I know you don’t speak the language but I’m certain you do know bonjour and merci!” which I loved.
Tho the security check around the town seems a bit sloppy given the circumstance.


UK – London
Good: People are unbelievably well mannered, cheap and quality shows
Bad: Transport is really expensive and often get packed(tube) or jammed(bus), food isn’t going anywhere
People speak, although I heard it really depends on region, utmost standard English. It’s like “From the creator of English!”
Manner standard is remarkably high. Only people speak loudly in tube is tourist group.
Suggest book Haymarket show every night, it’s really worth the money.
TIP: Ticketing office may struck a bargain for last minute booking.


Netherlands – Amsterdam
Good: “I love smell of weed in the morning”, cheap LOBSTER!
Bad: Doesn’t have much attraction, it’s fairly small city
Have Lobster every day, well worth the money.
Also one must try waffle, don’t be disappointed by its looking.


Trafalgar and Top Deck Tours

It was already some 20+ pages of itinerary so decided outsource some part of the planning and it was biggest mistake I made for this trip. Please do consider below before you book a guided tour:

1. THIS IS SINGLE BUS TOUR
The tour operators only use single bus to run entire tour, which means jumping between country to country will take 6-10 hours bus ride.
This might work if you use single country only tour such as 5 cities of Italy but 5 countries in 10 days type of program means about half of your tour will be spend on the road and is quite bumpy.

2. Your driver work during standard hour
What makes above worse is, bus driver only works during the day and need to take 30 minutes off every 2 hours. So every time when you depart and arrive from/to a city, you can’t really do anything on the day as pretty much everything is closed by then.
Considering you can easily get night time train, flight even drive hired car after 6-7PM otherwise, this is utter waste of time.

3. DLC..I mean, everything is optional
What the tour program provide for the marked price is pretty much ride, hotel, (some)meals and brief city tour and that’s it.
Everything else is optional, means extra $$$. For Trafalgar, doing all optional tours means you will pay approx. 50% more from initial price.

Lets Encrypt nginx SSL certificate renewal via cron

Add below as a cron entry.

#!/bin/bash
/path/letsencrypt/letsencrypt-auto --server https://acme-v01.api.letsencrypt.org/directory --renew-by-default -a webroot --webroot-path /webroot/ --email youremail --text --agree-tos --agree-dev-preview -d 7979.us -d www.7979.us auth
/etc/init.d/nginx reload

Then configure nginx SSL like below.

    listen 443 ssl;
    ssl_certificate /etc/letsencrypt/live/7979.us/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/7979.us/privkey.pem;
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_prefer_server_ciphers on;
    ssl_ciphers HIGH:!aNULL:!MD5:!3DES;


You may want to replace openssl default Diffie-Hellman (DH) key for enhanced security.
/usr/bin/openssl dhparam -out /etc/ssl/certs/dhparam.pem 4096

Then add below into nignx SSL config:
ssl_dhparam /etc/ssl/certs/dhparam.pem;

Q: What if I want to put SNI for all the domain including reverse proxy?
A: Put conditional redirect in nginx for ACME to point the same path and add sni with -d switch

location /.well-known/acme-challenge { alias /weboot/.well-known/acme-challenge; }
location / { return 301 https://$server_name$request_uri; }

Remember this will expose your domain structure in certificate.