I’ve been using CloudFormation YAML syntax for a while now with Ansible and the serverless framework which would convert the YAML to JSON before uploading the template. That already gave me the YAML advantages of e.g. code comments, not having to care about commas etc.
A few days ago, AWS announced native YAML support for CloudFormation templates, in addition to the existing JSON format.
And along with that they added new shorthand syntax for several functions.
Let’s go through a template which I created not only in order to get used to the new syntax :)
My current experiment is to build a serverless deploy pipeline (With AWS CodePipeline) which uses AWS Lambda for the build steps. One step includes to invoke NPM to build a static website out of JavaScript components (which would be deployed to an S3 bucket in a later step).
Ok, so let's go ahead and look what is actually installed in the preconfigured Node 4.3 env on AWS Lambda. First we want to find out if NPM is actually already installed. So we just create a new Lambda function which invokes a `find’ command, here is the used source code:
exports.handler = (event, context, callback) => {
var child_process = require('child_process'); console.log(child_process.execSync('find /usr -name npm -type f', {encoding: 'utf-8'}));
};
And, voila, we found something, here is the output:
/usr/local/lib64/node-v4.3.x/lib/node_modules/npm/bin/npm
So let's try to execute it!
console.log(child_process.execSync('/usr/local/lib64/node-v4.3.x/lib/node_modules/npm/bin/npm version', {encoding: 'utf-8'}));
And here is the output:
module.js:327 throw err; ^ Error: Cannot find module '/usr/local/lib64/node-v4.3.x/lib/node_modules/npm/bin/node_modules/npm/bin/npm-cli.js' at Function.Module._resolveFilename (module.js:325:15) at Function.Module._load (module.js:276:25) at Function.Module.runMain (module.js:441:10) at startup (node.js:134:18) at node.js:962:3 at checkExecSyncError (child_process.js:464:13) at Object.execSync (child_process.js:504:13) at exports.handler (/var/task/index.js:4:29)
Ok, that doesn’t look good, does it? Actually the ‘node_modules/npm/bin/node_modules/npm/bin/npm-cli.js’ part looks broken.
Ok, so my next step was to find the correct path to npm-cli.js, so I have a chance to call it without the apparently broken executable wrapper:
console.log(child_process.execSync('find /usr -type f -name npm-cli.js', {encoding: 'utf-8'}));
/usr/local/lib64/node-v4.3.x/lib/node_modules/npm/bin/npm-cli.js
So let’s try to call it directly:
console.log(child_process.execSync('node /usr/local/lib64/node-v4.3.x/lib/node_modules/npm/bin/npm-cli.js version', {encoding: 'utf-8'}));
gives us:
{ npm: '2.14.12', ... }
Yay! We got NPM working!
But NAY, it’s an old version!
So let's go ahead and try to install a newer version! Lambda gives us a writable /tmp
, so we could use that as a target dir. NPM actually wants to do much stuff in the
$HOME
directory (e.g. trying to create cache dirs), but it is not writable within a Lambda env.
So my “hack” was to set the $HOME
to /tmp
, and then install a newer version of NPM into it (by using the --prefix option
):
process.env.HOME = '/tmp'; console.log(child_process.execSync('node /usr/local/lib64/node-v4.3.x/lib/node_modules/npm/bin/npm-cli.js install npm --prefix=/tmp --progress=false', {encoding: 'utf-8'})); console.log(child_process.execSync('node /tmp/node_modules/npm/bin/npm-cli.js version', {encoding: 'utf-8'}));
Ok, NPM got installed and is ready to use!
npm@3.10.5 ../../tmp/node_modules/npm
The last step is to symlink the npm wrapperso it can be used without hassle. And actually many build systems seem to expect a working npm executable:
fs.mkdirSync('/tmp/bin'); fs.symlinkSync('/tmp/node_modules/npm/bin/npm-cli.js', '/tmp/bin/npm'); process.env.PATH = '/tmp/bin:' + process.env.PATH; console.log(child_process.execSync('npm version', {encoding: 'utf-8'}));
And here we go! Now it’s possible to use a up-to-date version of NPM within a Lambda function.
Some additional learnings:
child_process.execSync('rm -fr /tmp/.npm');
// ... npm install steps ...
Over the last years my Pocket reading queue got longer and longer. It actually dated back to stuff from 2013. Over the time a realized I would never ever be able to keep up with it again.
Some days ago I found out that Daniel (mrtazz) developed a nice tool named pocketcleaner which archives too old Pocket entries. I thought "Hey great, that's one solution to my problem, but how to execute it?". People who know me might already have an idea :) I don't like servers in terms infrastructure that I have to maintain. So I thought AWS Lambda to the rescue!
And here it is: An Ansible playbook which setups a Lambda function which downloads, configures and executes the Go binary. It can be triggered by a AWS event timer. No servers, just a few cents per month (maximum!) for AWS traffic and Lambda execution costs.
Inspired by the "JavaScript Equality Table" I created the same table for PHP:
!!! ATTENTION: Highly unstructured braindump content !!!
In an earlier first post I demonstrated a way to connect an external slave to a running RDS instance. Later then AWS added the native possibility to import and export via replication.
In my case, several problems popped up:
Better late then never. Here’s my little puppetconf 2013 recap.
Disclaimer: German.
Auch Aaron Swartz zu Ehren - veröffentliche ich meine Diplomarbeit "Wissensmanagement als integraler Bestandteil des Software Engineerings - Konzeption einer Vorgehensweise unter Einbindung agiler Modelle am Beispiel des Web 2.0-Unternehmens Jimdo".
In der Diplomarbeit wird untersucht, inwiefern die Anwendung agiler und Lean- Softwarentwicklungsmethoden wie Scrum, Software Kanban, Extreme programming und DevOps schon Wissensmanagement implizit "betreiben".
Hierzu werden die einzelnen Methodenbausteine (z. B. "Daily Standup" oder "Pair programming") vor dem Hintergrund des Wissensmanagement als Geschäftsprozess untersucht. Daraus wird ein Wissensmanagementmodell entwickelt, welches sich am Unternehmen Jimdo ausrichtet.
Vielen Dank an meine Frau Mila, Boris, Judith und Bracki, die fleißig und unermüdlich korrekturgelesen sowie wichtige inhaltliche Kritik geliefert haben!
While introducing rspec-puppet into a big and grown puppet codebase at Jimdo we needed to debug stuff and get more verbose output while writing the first tests. As the interwebs aren't very chatty about the topic, here for all the distressed googlers:
mysql> prompt (\u@\h) [\d]>
PROMPT set to '(\u@\h) [\d]>'
(debian-sys-maint@localhost) [cms]>
persistent in bashrc:
export MYSQL_PS1='(\u@\h) [\d]> '
Today I finally managed to set up an incremental backup for my workstation. What do we need? Well, nothing more than rdiff-backup, an opensource command line tool with all the powers you need.
it just backups. and your last backup is always available 1:1 at the destination (no strange storage formats etc., just dirs and files). Diffs and metadata are stored separately. So if you want a backup that does its job, is plain, is easy to restore, has no unneccessary features and "just works" than rdiff-backup is the right tool for you.
Here a small bash script I write to accomplish the mission:
#!/bin/bash
P=/home/soenke
DESTINATION=mydestionationserver.local
INCLUDE="
$P/Documents
$P/.gnupg
"
echo "$INCLUDE" | rdiff-backup --include-filelist-stdin --exclude $P/ $P/ $DESTINATION::/home/soenke/backup
Google search is now also possible via SSL. If you are paranoid like me you might wanna have it enabled by default - while there are several firefox plugins around there, here's how to do it on your own:
<Url type="text/html" method="GET"
template="http://www.google.com/search"
to<Url type="text/html" method="GET" template="https://www.google.com/search">
This tutorial is also valid for other OSes, but you may have to find the appropiate locations / directories yourself ;)
I've been using my own google search xml for ages anyway to change the default language to english because it's annoying to have auto-geo-autodetected language search results everytime esp. if
you search for technical things most of your time (Yes you could also set your preferred language in google, stay logged in all the time, let your cookies survive browser restarts etc, but that's
no option for me). To accomplish this, just add a param line to the xml file like this: <Param name="hl" value="en"/>
I just had the need for an Iterator that can sort itself by a user defined callback.
My special use case is that the DirectoryIterator of PHP does not sort the file list so it's pretty random. But my program logic relies on files being sorted by filename.
So here's the little class: