Jekyll Combining an External Folder into Posts
I use markdown to take notes. Previously, I wanted make a link to that folder and publish only some of those notes using Jekyll.
Then I realized that I could also do that for posts.
First, I tried linking the folder, filtering it, and then combining it into the posts collection. This didn’t work. The post was not being rendered.
I found that I had to filter the linked collection, then loop through it and add the posts one by one to the posts collection.
This is the end result:
# _plugins/filter.rb
module Filter
def self.process(site, payload)
site.collections['linked'].docs.select!{|x| x.data['slug'].include? 'blog.linked'}
now = Time.now
site.collections['linked'].docs.sort_by{|x| Time.parse(x.data['publish_on'])}.each do |x|
t = Time.parse(x.data['publish_on'])
if t <= now or site.config['future']
new_doc = Jekyll::Document.new(
x.path,
{site: site, collection: site.collections['posts']}
)
new_doc.read
new_doc.data['date'] = t
new_doc.data['draft'] = false
new_doc.data['categories'] = x.data['categories']
new_doc.data['description'] = x.data['desc']
x.data.delete 'desc'
new_doc.data['layout'] = 'post'
new_doc.data['slug'] = x.data['slug'].gsub("blog.linked.", "").split('.').last
new_doc.data['__coll'] = "posts"
site.collections['posts'].docs << new_doc
end
end
end
end
Jekyll::Hooks.register :site, :post_read do |site, payload|
Filter.process(site, payload)
end
What it does
I decided on blog.linked as the prefix for the notes that I wanted to publish. This removes any files that don’t follow that.
site.collections['linked'].docs.select!{|x| x.data['slug'].include? 'blog.linked'}
Then I loop through all the remaining posts. They need to be added in the order of their publishing date.
site.collections['linked'].docs.sort_by{|x| Time.parse(x.data['publish_on'])}.each do |x|
This check is to not pull in posts that are schedule in future. This could be improved to listen for the --future switch.site.config['future'] is the setting whether future posts should be published and can be set in the config.yml or in the command line using the --future.
t = Time.parse(x.data['publish_on'])
if t <= now or site.config['future']
Create a Jekyll::Document that references the site and sets the correct collection.
new_doc = Jekyll::Document.new(
x.path,
{site: site, collection: site.collections['posts']}
)
read the document contents:
new_doc.read
Then clean up the front matter.
new_doc.data['draft'] = false
new_doc.data['categories'] = x.data['categories']
new_doc.data['description'] = x.data['desc']
x.data.delete 'desc'
new_doc.data['layout'] = 'post'
new_doc.data['date'] = Time.parse(x.data['publish_on'])
new_doc.data['slug'] = x.data['slug'].gsub("blog.linked.", "").split('.').last
new_doc.data['__coll'] = "posts"
With the document created, it’s added into the posts collection.
site.collections['posts'].docs << new_doc
Finally register the hook to run after the files have been loaded but before render. It needs to be run on post_read but before render so that it runs before pagination.
Jekyll::Hooks.register :site, :post_read do |site, payload|
Filter.process(site, payload)
end
This way I don’t have to keep the notes in sync and any notes that I add will be automatically added to the site.
Update
I’ve also added the ability to fix links between files as well as correctly embed images.
new_doc.content.gsub!("", "")
new_doc.content.gsub!(/\[(.+)\]\(..\/(.+)\/(.+)\.md\)/, '[\1](/\3)')
I do still have to copy the images to the blog. At the moment I think this is a good thing because I make them smaller for the blog but the notes use the full resolution ones.
Another update
I’ve also added the ability to have parts of the markdown file excluded from being generated in Jekyll:
new_doc.content = new_doc.content.split("< !-- exclude -->").map{|x| x.split("< !-- include -->")}.select{|x| x.size > 1}.map{|x| x[1]}.join("")
Note the space in <!-- exclude-->. This is due to because the syntaxt highlighter plugin renders each term within a span which the interprets the rest of <!-- exclude--> as a comment.