ever2text: Migrating off of Evernote to Dropbox with Plain Text Files

ever2text: Migrating off of Evernote to Dropbox with Plain Text Files

tl;dr

ever2text converts Evernote exports of notebooks and files to pure text files, stored in Dropbox.

Why migrate off of Evernote?

Evernote recently announced they were limiting the free tier of accounts to only 2 devices, and also substantially increasing the pricing of all of their plans. You can read their announcement here.

I used Evernote on 5 devices, but I only used Evernote to take text notes. I didn’t use Evernote for images, PDFs, email support, or any of the other features included in the — now costlier — paid plans. I just use plain text notes.

I started working on a plan to move to a different platform for my notes. My goal was to get plain text files that I could use with any text editor, and on any device.

Migrating to Dropbox

I heavily use Dropbox, so Dropbox seemed like the best place to store all of my text notes. With Dropbox, I could access my notes on all of my devices and store them permanently in plain text files.

The Dropbox app for phones and tablets (both Android and IOS) also lets you create, search, and edit plain text files. This means I have no need for any additional apps on my phone and tablet devices for taking notes.

The problem with Evernote became: how do I export my all of my Evernote notes and notebooks to plain text files?

Exporting Evernote notes and notebooks to plain text files with ever2text

After some googling, I came across ever2simple which was created to export Evernote ENEX export files to SimpleNote format. It could also export to a directory, but it left me desiring more functionality, such as preserving the note’s title in the generated filename, and an option to export to pure text rather than a markdown formatted text file.

I decided to create a pure Evernote to text file converter: ever2text.

Ever2text converts Evernote ENEX exports to files while preserving the note title in the file name and the ability to choose either raw text or markdown for the formatting.

I did come across an issue with exporting Evernote notebooks. It appears you are unable to export all notebooks in to a single export file. I had a lot of notebooks I had created over the years, so I ended up having to export each Evernote notebook individually in to separate ENEX export files. You can export an evernote notebook by right-clicking on the notebook then clicking on “Export Notes”. Make sure to select the option to export as a file in ENEX format.

You can find ever2text in my github: https://github.com/nicholaskuechler/ever2text

Jekyll with Clean URLs Hosted at Rackspace Cloud Files

I’ve been using Jekyll to generate static web sites and then hosting them on the Rackspace Cloud Files CDN which uses Akamai’s content delivery network (CDN).

With Rackspace Cloud Files I have a CDN-enabled container and I have enabled my container to serve static web site files. This means I can use Rackspace Cloud Files with Akamai CDN to serve all my static web sites and I do not need to run or manage my own servers for web hosting. I simply use Cloud Files to store and serve my site. Some bonuses to using Cloud Files with CDN are my site is served to visitors very fast and my site can easily handle a very large number of visitors. Basically, my static site can handle web scale traffic.

What are Clean URLs?

I’m an advocate of using clean URLs, or human-readable URLs, in my sites. Clean URLs have many benefits:

  • Search engine optimization
  • Improved usability
  • Improved accessibility
  • Simplifies URLs
  • Easier to remember URLs
  • Do not contain implementation details of your site (Example: no php / html / asp / etc extensions on the URL)

Here’s an example of an un-clean URL:

http://www.domain.com/category/post-name-here.html

And here’s an example of a clean URL:

http://www.domain.com/category/post-name-here

Notice there is no .html and the URL looks better. Cleaner.

What is Jekyll?

Jekyll is a simple, blog aware, static site generator written in Ruby. It lets you create text-based posts and pages and a default layout that will be used across all of your posts. So you can easily change the look and feel of your site by modifying your default template and then re-generate your site, and the changes will be applied to all of your blog posts. Jekyll also generates static files that you can use on your CDN or host them yourself on your own server.

Jekyll does not create clean URLs by default, however. It will append .html to the file name and reference URLs with the .html suffix. Not ideal for a clean URL.

How To Get Clean URLs with Jekyll

I’m using a jekyll plugin which rewrites the file name and URL reference so that the html suffix is not included. It turns your blog-post.html file name in to “blog-post” without the .html extension.

To use Clean URLs with jekyll, you’ll need to set your permalink format in your jekyll _config.yml and use a jekyll plugin to generate your web site files without the .html extension.

Here is the _config.yml permalink structure I use for my site:

permalink: /:categories/:title

This will create a friendly URL in the form of: http://www.domain.com/articles/my-awesome-article

If you don’t want to display the category in the URL, you can change the permalink to:

permalink: /:title

And this will create a URL in the format of: http://www.domain.com/my-awesome-article

Check out the jekyll plugin I’m using on my github here: jekyll-rackspace-cloudfiles-clean-urls

Rackspace Cloud Files with Jekyll and Clean URLs

I came across another problem: Rackspace Cloud Files does not know what type of file “blog-post” is as there is no file extension on it. When you browse to my CDN-hosted site to a clean URL, your browser would try to download the file instead of rendering it as html. The reason is that Cloud Files can’t peer inside the file and see that it’s all HTML code and apply the correct content type. I needed to manually set the content type myself and tell Rackspace Cloud Files that “blog-post” is type “text/html” so that a web browser can properly display it.

In order to solve this problem I have written a python helper script to apply the “text/html” content type automatically for my jekyll generated sites. My python helper script will upload my site to Rackspace Cloud Files for me and check the files it has uploaded to see if they are HTML files or not. If an HTML file is found, the python helper script will tell Cloud Files it is type “text/html”, allowing Cloud Files to properly display the html to a browser.

Download my Cloud Files / jekyll helper script from my github: jekyll-rackspace-cloudfiles-clean-urls

Auto-scale Rackspace Cloud servers with Fabric and Celery

Auto-Scaling Celery Workers with Rackspace Cloud Servers

Python logo
Python

I have been working on an automated stock trading system for some time. Part of my automated trading system involves a lot of number crunching and calculations, such as for technical analysis and neural networks. Processing large amounts of data for thousands of stock tickers can be very time consuming and take a significantly long time to fully process the data. I have begun creating additional worker nodes so that I have more processing power available in order to complete the data processing and number crunching much faster.

I have created some tools to help me manage these additional worker nodes and simplify the scaling up and scaling down of the data processing workers. My tools will automatically scale up worker nodes using the Rackspace Cloud Servers API and then start my python celery workers to crunch the data and process the tasks in my rabbitmq queue. When the processing has been completed and the tasks in queue are zero, the auto-scaling script will spin down and destroy the worker cloud servers.

The auto scaling tool checks my rabbitmq queue size and if there are a large amount of tasks in the queue, it will create a number of cloud servers using the Rackspace Cloud API and an image template I have created and saved at Rackspace Cloud.

Basically it works like this:

  • Each worker instance is built from a template image, so it has the exact same packages and code base.
  • If the queue size is very large, then create a bunch of workers.
  • If the queue size is 0, then delete the additional workers, to save money by removing instances we’re not actively using.

I’m using Fabric for python to create the celery worker servers. The celery workers then work on tasks in a RabbitMQ messaging queue. Fabric is a very powerful tool that can run commands automatically on the newly created servers. I am also using the python-cloudservers python package to interface with Rackspace Cloud API and create the servers. After the servers are created, I’m using rsync to copy over my code base to the newly created cloud servers. Fabric then starts up the celery worker daemon on the newly created worker nodes. The celery daemon on the worker then takes care of the rest and starts processing the tasks.

The script I’m using to auto-scale up and and down is a custom script I have written. It runs the auto-scale.py via a crontab entry on my primary/master processing server.

You can find my scripts and code at github:

My crontab entry for auto-scale.py:

*/5 * * * * /usr/bin/python /opt/codebase/autoscale-rackspacecloud-fabric-celery/auto-scale.py >> /tmp/auto-scale.log

Download VMware Appliances and Templates

Here are 2 places you can download ready to use VMware appliances and operating system templates:

You’ll be able to find many different appliances and images. Some of the more popular downloads are: Linux (Fedora, Ubuntu, Debian), FreeBSD.

Also check out VMware HOWTOs also at thoughtpolice.

MarketClouds.com: Stock Market Trading Clouds

Indroducing Market Clouds – www.marketclouds.com – stock market activity and popularity indicator using clouds.

I have developed a new tool to assist stock market traders and investors in finding the most popular and active companies traded on all of the stock market exchanges: Market Clouds.

Continue reading MarketClouds.com: Stock Market Trading Clouds