Org Babel API

Babel is part of Emacs Org mode which allows for interactive code blocks within a text document.

This is what a text document containing Org blocks looks like.  Below
is a shell block which installs some Python libraries.

#+begin_src sh :results output :exports both :eval never-export :session *worg-babel*
guix install python-matplotlib python-numpy
#+end_src

#+RESULTS:
#+begin_example
The following package will be upgraded:
   python-numpy (dependencies or package changed)

The following derivation will be built:
  /gnu/store/yh1wjfmy4jy75wswkdsfa0lrfw7jrxs9-profile.drv

building CA certificate bundle...
listing Emacs sub-directories...
building fonts directory...
generating GdkPixbuf loaders cache...
generating GLib schema cache...
creating GTK+ icon theme cache...
building cache files for GTK+ input methods...
building directory of Info manuals...
building XDG desktop file cache...
building XDG MIME database...
building profile with 127 packages...
#+end_example

Now that we've installed the Python package, let's run it.

#+name: python-array
#+begin_src python :results value :session *worg-python*
import numpy as np

a = np.arange(15).reshape(3, 5)
a
#+end_src

#+RESULTS: python-array
: array([[ 0,  1,  2,  3,  4],
:        [ 5,  6,  7,  8,  9],
:        [10, 11, 12, 13, 14]])

For fun, let's take this result and pass it to the shell.

#+begin_src sh :results output :exports both :eval never-export :var x=python-array
echo $x | sed 's/],/],\n\t/g'
#+end_src

#+RESULTS:
: array([[ 0, 1, 2, 3, 4],
:        [ 5, 6, 7, 8, 9],
:        [10, 11, 12, 13, 14]])

It's just text, so we can pass it around.

#+begin_src scheme :results output :var x=python-array
(display "This is from Guile Scheme:\n")
(display x)
#+end_src

#+RESULTS:
: This is from Guile Scheme:
: array([[ 0,  1,  2,  3,  4],
:        [ 5,  6,  7,  8,  9],
:        [10, 11, 12, 13, 14]])

At the time of writing, the Org Babel API is only described in three places:

1. ob-template.el,
2. the email thread linked to in the Worg docs, and,
3. the source code itself.

The functions that need to be (or even can be) defined aren't really laid out anywhere, nor are the ways they interact explained in one place. This document aims to change that.

NOTE At some point I'll merge this into Worg¹. However, this is still a draft and I don't feel it's right to host it there yet. These are my active notes and people probably expect Worg to be somewhat more complete.

Org Babel is defined by a collection of Emacs Lisp files. Babel files begin with "ob-" (for "Org Babel") and are found in the lisp/ directory of the Org project². The primary file is ob-core.el.

ls $ORGDIR | grep "ob-" | grep "el$" | head -n 10 && echo "..."

At the time of writing, Babel is useable with more than 40 different languages.

ls $ORGDIR | grep "ob-" | grep "el$" | wc -l

Babel is no longer maintained by the original author. Instead, extensions have individual maintainers. Each extension varies in quality and consistency. Some see regular updates whereas others don't seem to have changed much.

It's convention in Elisp to choose a prefix to distinguish functions and variables between packages. The name "Org Babel" implies the package prefix "org-babel-"³. This is the prefix used within Babel as well as other Org extensions (e.g. Org Links, Org Export Framework, etc.). That is, symbols defined in ob-<lang>.el don't use a "ob-<lang>-" prefix–they use "org-babel-". It's impossible to distinguish Org Babel symbols from other Org symbols by name alone.

Babel works by defining certain functions that ob-core.el expects.

If we compare ob-template.el and ob-core.el, we get the following in no particular order:

Functions from ob-core.el not in ob-template.el, but still used in some ob-template.el derived languages (e.g. ob-shell.el):

• org-babel-header-args:template
• org-babel-load-session:template
• org-babel-variable-assignments:template

Important functions to understand for any extension that uses a comint (all of them?):

• org-babel-comint-wait-for-output

Arguments: body, params

Returns: emacs-lisp containing the results of evaluating body in template language.

The function called to actually evaluate a code block. It is defined in an "ob-" file and gets called by org-babel-execute-src-block.

The org-babel-execute:template function will evaluate the body of the source code and return the results as emacs-lisp depending on the value of the :results header argument:

• :results output means that the output to STDOUT will be captured and returned
• :results value means that the value of the last statement in the source code block will be returned

The ob-template.el file claims that "the most common first step in this function is the expansion of the PARAMS argument using org-babel-process-params." This was probably true when that was written. At the time of writing, however, only four of the files in the org-babel suite use it.

Please feel free to not implement options which aren't appropriate for your language (e.g. not all languages support interactive "session" evaluation). Also you are free to define any new header arguments which you feel may be useful – all header arguments specified by the user will be available in the PARAMS variable.

Called by org-babel-execute:template.

This is cruft. It's only defined in ob-template.el. All it does is this:

(defun org-babel-template-var-to-template (var)
  "Convert an elisp var into a string of template source code
specifying a var of the same value."
  (format "%S" var))

Surely this doesn't deserve a whole function.

This is cruft. It is defined in ob-template.el but not used anywhere, in that file or elsewhere.

"Expand variables in PARAMS and add summary parameters."

Gets called by ob-tangle.el in the org-babel-effective-tangled-filename function.

Documented in org-babel-default-header-args:

This is a list in which each element is an alist. Each key corresponds to a header argument, and each value to that header's value. The value can either be a string or a closure that evaluates to a string.

It's important to note that org-babel-default-header-args is actually different from org-babel-default-header-args:template. org-babel-default-header-args is only used for inline source blocks:

(defvar org-babel-default-inline-header-args
  '((:session . "none") (:results . "replace")
    (:exports . "results") (:hlines . "yes"))
  "Default arguments to use when evaluating an inline source block.")
(put 'org-babel-default-inline-header-args 'safe-local-variable
     (org-babel-header-args-safe-fn org-babel-safe-header-args))

The variable org-babel-default-header-args:template is used by org-babel-get-src-block-info. During block evaluation, the language is grabbed from the source block definition and constructed to form the language specific variable. For example, if a "shell" block is evaluated, the corresponding variable org-babel-default-header-args:shell is constructed and referenced.

Called by the ob-core.el function org-babel-load-in-session. That function handles header args, mainly no-web expansion. It then starts the comint process and opens the associated process buffer. The org-babel-load-in-session function is, basically, called by the "org-metaup-hook" which is attached to M-up by default. Otherwise, this function is never called. Also, AFAICT, this behavior is not documented anywhere.

For ob-shell.el, this function is implemented, but only for the "shell" language. For example, press M-up when in the following block. The body will be inserted in a new process buffer "my-shell".

#+begin_src shell :session my-shell
echo "hello, world!"
#+end_src

#+begin_src sh :session my-sh
echo "hello, world!"
#+end_src

#+begin_src bash :session my-bash
echo "hello, world!"
#+end_src

Used during tangle (ob-tangle.el) and eval.

It is used as an argument for org-babel-expand-body:generic.

It is used in org-babel-prep-session:template

"Return list of template statements assigning the block's variables."

Don't be surprised if there is undocumented behavior defined within an ob file.

The following were undocumented features of ob-shell.el:

• :stdin passes org reference (#+name) as stdin
• :cmdline with :shebang runs body with given command-line arg(s)
• :cmdline without :shebang runs body with shell-file-name and command-line arg(s)
• org-babel-load-session:shell (see org-babel-load-session:template)

To quote (org) Structure of Code Blocks,

A source code block conforms to this structure:

    #+NAME: <name>
    #+BEGIN_SRC <language> <switches> <header arguments>
    <body>
    #+END_SRC

The :var parameter has the form

(:var name . value)

where "value", the cdr, should be a cons cell whose car is the name of the variable "as a symbol" and whose value is the value of the variable.

That is,

(:var name . (name-symbol . "it's value"))

• https://lists.gnu.org/archive/html/emacs-orgmode/2015-09/msg00487.html