[Work in Progress] Meteor+Docker+Flow

ArmandDu -

I've been working with Docker, React and Meteor for a while now. These tools are quite popular, but having them play together can sometimes be a huge hassle.

As I'm learning more about each of them, I'm building some tools to help ease the hassle.

This post will be a work in progress where I put my issues and my current solutions.

I don't have comments enabled in this blog yet. If you are reading this and want to comment or contribute to this WiP, you can post an issue or fork everything here: armanddu/meteor-docker-flow.

Issue: Flow Cannot resolve module

Meteor builds your packages under the .meteor/local folder.
When doing something like import Package from "meteor/package", Meteor will know where it resolves.

Unfortunately, when using flow to check your files, by default it will try to resolve the module from "node_modules".. which will fail.

To solve the issue, you need to dig into the .flowconfig file. More precisely, the [options] section and its module.name_mapper option.

Once you know how to add custom module mappers, you need to know where the modules are and the different ways to import a Meteor package.

Long story short, here are the options you need to pass:

# .flowconfig

[options]

module.name_mapper='^\/\(.*\)$' -> '<PROJECT_ROOT>/\1'
module.name_mapper='^meteor\/\(.*\):\(.*\)$' -> '<PROJECT_ROOT>/.meteor/local/build/programs/server/packages/\1_\2'
module.name_mapper='^meteor\/\(.*\)$' -> '<PROJECT_ROOT>/.meteor/local/build/programs/server/packages/\1'
module.name_mapper='^meteor\/\(.*\)$' -> '<PROJECT_ROOT>/.meteor/local/build/programs/web.browser/packages/\1'

Issue: Meteor doesn't have type checking

If you've read my blog this far, you know I'm using the flow static type checker.

One nice feature you can use with flow is flow-typed which, according to the readme,

[...] is a repository of third-party library interface definitions for use with Flow.

It lets you install the library definitions of the modules you installed with npm or yarn.

The big "bonus" is that if the library definition doesn't exist, flow-typed will create a file (ie: my_module_vxxx.js) and create a default definition for every export in that module.

Basically, for every export in your third-party module, you will have:

# flow-typed/npm/my_module_vxxx.js
declare module "my_module" {
  declare module.exports: any;
}

Which is great because:

  1. You get the warning when trying to import something not exported, and
  2. You can edit the file yourself if you know how to define the types.

Unfortunately, you don't install meteor with npm install. Consequently, flow-typed doesn't have the definition for Meteor or its many packages... It's up to you (and me too) to define them yourself. Enjoy!

I have found one project that created a basic library definition for Meteor's Mongo here: Sanjo/meteor-flow-types. It is inspired by the TypeScript's library definition for Meteor.

Thanks to Sanjo and the TypeScript guys, I now have a skeleton of Meteor and Mongo Collection library definition!

It's basically a fork of Sanjo's that I'm improving when I need/have time to do. Currently, you'll find it here in my repo ArmandDu/meteor-docker-flow under flow-typed/.

You can actually use it for some parts. When it will be complete enough, I think I'll give Sanjo a pull request.

Conclusion

TL DR: you currently have to write your own library definitions for Meteor & Co packages.

I'm writing my library definition and I'm sharing it!

Issue: It's a hassle do depend on open-source Dockerfiles to build a docker image.

Once the coding is done I used to spend about 1 hour to build and deploy my app.

At first I was using Meteor-up but it wasn't really "lightweight" nor easy to move.

Now I have a docker swarm and I'm planning to setup some CI/CD with Gitlab, so I need a more reliable and lightweight way to build my Meteor project and deploy it to my swarm.

The obvious answer is to use some Dockerfile, build a docker image and publish it to a registry... which I do with all my other projects.

But of course, that would too easy - so,how do I do that with Meteor?

At first, I identified that I didn't want to build my app locally and copy the bundle to a docker image because I learned to build an app in its final environment.
So I did some searching and found meteorhacks/meteord... oh, deprecated! But it says to use jshimko/meteor-launchpad. Which works! (Once you get the hang of the versions)

Well, in my case worked...

There were a couple of issues that I didn't have the courage or the luck to find a solution for:

1. It's super long to build.

Most of the instructions are "onbuild" including this:

# install all dependencies, build app, clean up
ONBUILD RUN cd $APP_SOURCE_DIR && \
  $BUILD_SCRIPTS_DIR/install-deps.sh && \
  $BUILD_SCRIPTS_DIR/install-node.sh && \
  $BUILD_SCRIPTS_DIR/install-phantom.sh && \
  $BUILD_SCRIPTS_DIR/install-graphicsmagick.sh && \
  $BUILD_SCRIPTS_DIR/install-mongo.sh && \
  $BUILD_SCRIPTS_DIR/install-meteor.sh && \
  $BUILD_SCRIPTS_DIR/build-meteor.sh && \
$BUILD_SCRIPTS_DIR/post-build-cleanup.sh

AKA: install base dependencies, Node, PhantomJS, Mongo, Meteor, then build the Meteor app and clean up.

It's understandable, but long. Plus, when it fails, it hurts.

2. It fails when you have local meteor packages.

I haven't looked too much into that, but it's quite easy to create your own Meteor package and I think your Meteor local env caches your installed and custom made packages. I suspect that because, when I ran the magical command meteor add armanddu:my-super-package (which is not deployed to Atmosphere), Meteor found it and installed it. One more suspicion was that, when I edited my package, my Meteor app that used the package was automatically and magically updated!

Anyway, that's not an issue - the issue is that, while Meteor knows where my code is on my harddrive and how to use it, docker doesn't. So when I tried to build my app with Meteor-launchpad, when it reached the second last instruction of the RUN command ("build-meteor"), the Meteor installed in the docker image was unable to find the package, and... the build failed... after way too long!

Solution ?

After that last issue, I was way too lazy to try to run the build and wait 15 minutes to see if it could find my package then. So I decided to code my own Dockerfile (which I should have done way earlier).

After checking a bit, Meteor build will just bundle the project, you still need do some npm install and env setting before running your app.

Sooo... why not bundle the app locally, add that to the docker image and install the dependencies inside the docker?

This is my current solution:

 #Dockerfile
 FROM	node:8

RUN	mkdir /app
WORKDIR	/app

ADD	/build/bundle	.
RUN	(cd programs/server && npm install)

ENV	PORT=3000
EXPOSE	3000

CMD	["node", "main.js"]

#package.json

{
  ...
  "version": "0.0.1",
  "scripts": {
    "start": "meteor run --settings dev-settings.json",
    ...
    "deploy": "docker-deploy",
    "build": "meteor build --directory build/"
  },
  "config": {
    "docker": {
      "imageName": "my.registry.com/my_image_name"
    }
  },
  ...
  "devDependencies": {
    "docker-deploy": "0.0.2"
  }
}

This is currently a lazy solution that works. I'll improve and update my Dockerfile when I have time but what's to take is:
I bundle my app locally, that way my local Meteor knows its stuff and builds nicely. When it's good, I run my docker build and docker push commands and that's it!

As I don't have to install Meteor or other crazy stuff, it's way easier and faster!

I'm using djskinner/docker-deploy because it's nice and I'm lazy.
It's a script that will run your docker build ... and docker push ... commands. It uses the version and configuration of you package.json file. So I have to do it once!

Here are the commands I run now:

$ #edit some code...
$ #git commit and stuff

$ npm version <my_app_version>
$ npm run build
$ npm run deploy

$ #currently ssh to server and deploy new version
$ #in the near future: let my .gitlabci file do the job

Conclusion

TL DR: using Meteor with Docker can be slow and painful, so I decided to bundle the app locally and build and run the bundle in a Docker image. I also used my package.json to configure my build and deploy process.

You can find the Dockerfile and an example package.json here : Armanddu/meteor-docker-flow

Issue: I'm forced to hide (.folder) the code I don't want in the bundle.

When developing using Meteor, it will watch, build and bundle everything in you working directory.

If you want some code, or config or files that you don't want in the bundle, you need to hide it by appending a . in front of your file or folder.

It's stupid, but for flow-typed you theorically need to do this:

$ mkdir .flow-typed
$ flow-typed install --libdefDir .flow-typed

Which you might forget fast since you (maybe?) don't install packages every two minutes.

You should have seen another issue in the previous section... I'm running meteor build --directory build/ ! That means Meteor might (will?) bundle my bundle in my next bundle...
Well you see the issue. Meteor even tells you when building:

WARNING: The output directory is under your source tree.
Your generated files may get interpreted as source code!
Consider building into a different directory instead
meteor build ../output

The solution "hide your file or folder" is actually the first answer you get when you look for it. But actually, since v1.5.2.1, Meteor supports a .meteorignore! You can use it like your .gitignore or .dockerignore !

I felt a bit stupid finding about that because I haven't even found about that in the doc... Just some lucky googling duckduckgoing...

So now that I've made sure to ignore my build/ folder, I can happilly bundle my app where I want and not in some .build or ../build.

Conclusion

TLDR: you can use .meteorignore since V1.5.2.1! Ignore all the files!

You can check my default .meteorignore there: Armanddu/meteor-docker-flow