All examples By author By category About

thedod

Full window Github gist

Hyde Park - githubless gh-pages with gitolite

###Hyde Park - githubless gh-pages with gitolite

 ________________________________________ 
< Content-type: text/plain. You feel me? >
 ---------------------------------------- 
        \   ^__^
         \  (OO)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

Hyde Park is a git-triggered jekyll [etc.] static site generation bot (you push - it builds).

There are many complex ways to do this, but IMHO — this one's simple (from experience) 😉.

It is meant for internal use inside organizations where you want to collaborate on tomorrow's content without the risk of it leaking too soon. (think political campaign sites ;) ).

Front end (what the 99% non-geeks get to see) is a static site with SSL and basic auth (if you want to read, you nead a user/password). Backend is gitolite (if you want to collab, you send an ssh key).

hydepark.py is the cgi script that gets trigerred by git's post-receive hook when you push. See below how to create such a hook and tell gitolite which repos need to trigger it, but what the hook does is a GET to the cgi sctipt with something like ?repo=bob/howto&branch=refs/heads/preview

The cgi script checks config.py to see whether the branch bob/howto/refs/heads/preview has a build script it should trigger, and if it does - launches it.

You need to copy config.py.example to config.py and edit it.

deploy-jekyll.sh is good for simple jekyll sites.
deploy-bundle.sh is for sites that need a special bundler environment.

For security reasons, the script doesn't run bundle install automatically (the line is commented out). Best is to run it manually at $DEPLOYDIR, after triggering the git hook for the first time (in order to create that folder).

env-jekyll is a tweak to setup a machine-specific environment so that jekyll can run (e.g. if you've installed your personal copy of jekyll).

If you have such a problem, copy env-jekyll.example to env-jekyll and edit it.

Another tweak is _config-hydepark.yml that gets copied to _config.yml before we build. This lets you define site.baseurl and other deploymet-specific parameters.

`.htaccess` files

We can't allow users to push .htaccess files because then they (or whoever steals their notebooks) can enable cgi-bin and run arbitrary code. On the other hand, we want to let users control who gets read access to their sites. What we do is make sure .htacess files are handled outside the scope of the git repo.

First, we need an .htaccess file for the root hydepark folder. You can user .htaccess.example as a basis. You also need to create the .htpasswords file using the htpasswd utility. It should at least contain the user githook (which is what the gitolite server uses in order to trigger hydepark.py), but you should probably have at least one additional user for yourself 😉.

This takes care of the root folder. Now lets see what happens at specific site folders.

The default: intranet-wide permissions

You (the admin) prepare a .htaccess.default file that gets copied by the deploy-*.sh scripts into the root folder of a site after building (and removing all existing .htaccess files). You can use .htaccess.default.example as a basis.

The Require valid-user line means that whoever has a user/password to the hydepark's web server can view the site (if they know the URL, but since some sites are forks of public ones, it's often easy to guess).

Restricting access to specific users

If Alice wants to only allow alice and bob access to alice/myproject/preview, you can copy .htaccess.alice.myproject.preview.example to .htaccess.alice.myproject.preview, and you're done (it contains the line Require user alice bob 😉). The deploy-*.sh script will find that file and use it as the site's .htaccess (instead of using .htaccess.default).

_plugins

By default, Jekyll looks for plugins in ./_plugins but this would let users run arbitrary Ruby code, so — as you can see at deploy-*.sh — we do the Jekyll builds with -p $SCRIPT_DIR/_plugins. This means you should create _plugins/ under this folder (even if you leave it empty), and you can put there plugins you trust aren't malware 😉.

Gitolite side

Create local/hooks/repo-specific/hydepark-hook with chmod 700 containing something like:

#!/bin/sh
BRANCH=`sed 's/.* //'`
echo Doing Hyde Park build for $GL_REPO/$BRANCH
curl "https://USER:PASSWORD@example.org/hydepark/hydepark.py/?repo=$GL_REPO&branch=$BRANCH"

See here how you can manage hooks as part of the gitolite-admin repo.

For each project that has one or more branches that need rendering on push, do something like this at gitolite.conf:

repo alice/howto
    # ...
    R       = hydepark
    option hook.post-receive = hydepark-hook

Note that the hydepark user is verified with a public SSH key, and the user running the hydepark.py cgi (i.e. the web server's user) should have the corresponding private key at its ~/.ssh/.

When you push, you're supposed to see the result of the hydepark.py script on the console:

No deployment for "alice/howto/refs/heads/master"

On the web server box, you need to define which branch[es] get[s] rendered and how by editing config.py:

DEPLOYMENT = {
    # ...
    'alice/howto/refs/heads/master':'deploy-jekyll.sh',
}

That's it: simple things are [relatively] simple, and complex things are [quite] possible ;)

<h3>Hyde Park - githubless gh-pages with gitolite</h3>

<pre><code> ________________________________________ 
&lt; Content-type: text/plain. You feel me? &gt;
 ---------------------------------------- 
 \ ^__^
 \ (OO)\_______
 (__)\ )\/\
 ||----w |
 || ||
</code></pre>

Hyde Park is a git-triggered jekyll [etc.] static site
generation bot (you push - it builds).

There are many complex ways to do this, but IMHO &mdash; this one's
simple (from <a href="https://swatwt.com/howto">experience</a>) &#128521;.

It is meant for internal use inside organizations
where you want to collaborate on tomorrow's content
without the risk of it leaking too soon.
(think <a href="https://elections2015.no2bio.org">political campaign sites</a> ;) ).

Front end (what the 99% non-geeks get to see) is a static site
with SSL and basic auth (if you want to read, you nead a user/password).
Backend is gitolite (if you want to collab, you send an ssh key).

<code>hydepark.py</code> is the cgi script that gets trigerred by
git's post-receive hook when you push. See below how to
create such a hook and tell gitolite which repos need to trigger it,
but what the hook does is a GET to the cgi sctipt with something like
<code>?repo=bob/howto&amp;branch=refs/heads/preview</code>

The cgi script checks <code>config.py</code> to see whether the branch
<code>bob/howto/refs/heads/preview</code>
has a build script it should trigger, and if it does - launches it.

You need to copy <code>config.py.example</code> to <code>config.py</code> and edit it.

<ul>
<li><code>deploy-jekyll.sh</code> is good for <a href="https://solo.chibi.io/">simple</a> jekyll sites.</li>
<li><code>deploy-bundle.sh</code> is for <a href="https://github.com/ellekasai/shiori#readme">sites</a>
that need a special <a href="https://bundler.io/">bundler</a> environment.

For security reasons, the script doesn't run <code>bundle install</code>
automatically (the line is commented out).
Best is to run it manually at <code>$DEPLOYDIR</code>, after 
triggering the git hook for the first time (in order to create
that folder).</li>
</ul>

<code>env-jekyll</code> is a tweak to setup a machine-specific
environment so that jekyll can run (e.g. if you've
installed your personal copy of jekyll).

If you have such a problem, copy <code>env-jekyll.example</code> to <code>env-jekyll</code> and edit it.

Another tweak is <code>_config-hydepark.yml</code> that gets copied to
<code>_config.yml</code> before we build. This lets you define <code>site.baseurl</code>
and other deploymet-specific parameters.

<h4><code>.htaccess</code> files</h4>

We can't allow users to push .htaccess files because then they
(or whoever steals their notebooks) can enable cgi-bin and run
arbitrary code. On the other hand, we want to let users control
who gets read access to their sites. What we do is make sure
<code>.htacess</code> files are handled outside the scope of the git repo.

First, we need an <code>.htaccess</code> file for the root hydepark folder.
You can user <code>.htaccess.example</code> as a basis.
You also need to create the <code>.htpasswords</code> file using the
<a href="https://httpd.apache.org/docs/current/programs/htpasswd.html">htpasswd</a>
utility. It should at least contain the user <code>githook</code>
(which is what the gitolite server uses in order to trigger <code>hydepark.py</code>),
but you should probably have at least one additional user for yourself 
&#128521;.

This takes care of the root folder. Now lets see what happens
at specific site folders.

<h5>The default: intranet-wide permissions</h5>

You (the admin) prepare a <code>.htaccess.default</code> file that gets
copied by the <code>deploy-*.sh</code> scripts into the root folder of
a site after building (and removing all existing <code>.htaccess</code> files).
You can use <code>.htaccess.default.example</code> as a basis.

The <code>Require valid-user</code> line means that whoever has a user/password
to the hydepark's web server can view the site (if they know the URL,
but since some sites are forks of public ones, it's often easy to
guess).

<h5>Restricting access to specific users</h5>

If Alice wants to only allow <code>alice</code> and <code>bob</code> access to
<code>alice/myproject/preview</code>, you can copy
<code>.htaccess.alice.myproject.preview.example</code> to
<code>.htaccess.alice.myproject.preview</code>, and you're done
(it contains the line <code>Require user alice bob</code> &#128521;).
The <code>deploy-*.sh</code> script will find that file and use it
as the site's <code>.htaccess</code> (instead of using <code>.htaccess.default</code>).

<h4>_plugins</h4>

By default, Jekyll looks for plugins in <code>./_plugins</code> but
this would let users run arbitrary Ruby code, so &mdash;
as you can see at <code>deploy-*.sh</code> &mdash; we do the Jekyll
builds with <code>-p $SCRIPT_DIR/_plugins</code>. This means you
should create <code>_plugins/</code> under this folder (even if
you leave it empty), and you can put there plugins
you trust aren't malware &#128521;.

<h3>Gitolite side</h3>

Create <code>local/hooks/repo-specific/hydepark-hook</code>
with <code>chmod 700</code> containing something like:

<pre><code>#!/bin/sh
BRANCH=`sed 's/.* //'`
echo Doing Hyde Park build for $GL_REPO/$BRANCH
curl "https://USER:PASSWORD@example.org/hydepark/hydepark.py/?repo=$GL_REPO&amp;branch=$BRANCH"
</code></pre>

See
<a href="https://rdkls.blogspot.com/2014/02/gitolite-post-commit-hooks-for-jenkins.html">here</a>
how you can manage hooks as part of the <code>gitolite-admin</code> repo.

For each project that has one or more branches that need rendering on push,
do something like this at <code>gitolite.conf</code>:

<pre><code>repo alice/howto
 # ...
 R = hydepark
 option hook.post-receive = hydepark-hook
</code></pre>

Note that the <code>hydepark</code> user is verified with a public SSH key, and the user running the <code>hydepark.py</code> cgi (i.e. the web server's user) should have the corresponding private key at its <code>~/.ssh/</code>.

When you push, you're supposed to see the result of the <code>hydepark.py</code> script on the console:

<code>No deployment for "alice/howto/refs/heads/master"</code>

On the web server box, you need to define which branch[es] get[s] rendered and how by editing <code>config.py</code>:

<pre><code>DEPLOYMENT = {
 # ...
 'alice/howto/refs/heads/master':'deploy-jekyll.sh',
}
</code></pre>

That's it: simple things are [relatively] simple, and complex things are [quite] possible ;)