No more version history, now we have just the contents of the current version.
Editing a definition no longer changes the order in which definitions load.
This should make repos easier to browse, and more amenable to modify. You don't need driver.love anymore. And a stable order eliminates some gotchas. For example:
using driver.love, define Foo = 3
in a definition
define Bar = Foo + 1
edit and redefine Foo = 4
Before this commit, you'd get an error when you restart the app. Definitions used to be loaded in version order, and editing a definition would move it to the end of the load order, potentially after definitions using it. I mostly avoided this by keeping top-level definitions independent. It's fine to refer to any definition inside a function body, we only need to be careful with initializers for global variables which run immediately while loading.
After this commit you can still end up in a weird state if you modify a definition that other later definitions use. In the above example, you will now see Foo = 4 and Bar = 4. But when you restart, Foo = 4 and Bar = 5. But that's no more confusing than Emacs's C-x C-e. It's still a good idea to keep top-level definitions order-independent. It's just confusing in a similar way to existing tools if you fail to do so. And your tools won't tend to break as badly.
Why did I ever do my weird version history thing? I think it's my deep aversion to risking losing any data entered. (Even though the app currently will seem to lose data in those situations. You'd need to leave your tools to find the data.) Now I rely on driver.love's undo to avoid data loss, but once you shut it down you're stuck with what you have on disk. Or in git.
I also wasn't aware for a long time of any primitives for deleting files. This might have colored my choices a lot.
MFBZCXOUHS7ELP3SRTA7VDO32CUSRDIX44LXSMBPIAE5JXKFWXOQC
Z4THEMHVT7XSBX7EBPOIIXBQQLCHYFWGYNHWU5PY276D33HR6I4AC
4ZNQND6BQT7FCVVZV6O5LLWJCVBSU63I4H3LIZO4ROYUPYDXSDHQC
HQIOEEOTCMJR6MC7JFRBFE6ZX3TDTFRCZUH6ZYI3EWG3H7BE4PCQC
BZDHAD6RTFC5QMKGFWH45BZQF5X43CXVWMABMSFXM3H7LF3XVWEQC
Y7QULWZRH3M3TDNHVOBPDPRT5GTPX2BY3IY2FBUUDOH53CCBB6IAC
HERSSH5364OR37WXCNXKWWMJPZXKQXRO3PCST5X2C5UOEKMNXKVQC
FJFW3QQPLBDE2UR6O7UBY4DBN5SD3CBRQL7IM4IGCQ23LWLLWYKQC
57HKHZ7Z4QSCS6X35H5WZ5Y4MALGLYAPMUTP25BTU2MYTO6HOLXAC
K4SFKQ7VF6AYFJJE7GVEBGCSOYBCT3A7FM4YCO3ODOU5BI4KJ7FAC
R5QXEHUIZLELJGGCZAE7ATNS3CLRJ7JFRENMGH4XXH24C5WABZDQC
2JXGTZZXVGGB7RIDJPTJSF4V7OM5Y2MC7GPN5N4QZTYDYDZ2HKYQC
DBNRXNIRF3JNGMSP62USALHVA6GVWMAIUBRAD5FQIOTG5VR6RVRQC
5TDFTJBI6I633A4W2NWMMP3YRYOTLRSBJU7XBU5H45UETXSPGSBQC
GMDEH4RHOR6DJABQ5WVGY4GVUU3QPWFTPFUHYUOULTVJZA2FTPAQC
5WYBV55CRVPFXFA7QXTMFJG3653NGMLYUMNGN2Q7X2Q2XRUWZNFQC
OLNYTLSTFOVYVBR3SBIYUDFJZDU6FOTFO2S5RFEVEP4D5HIW2LZQC
KSM4V7K5PMMCSF5JTVQE7BKLXJLYV6CF4VKUWGWJ6CNQMPKPOVUAC
LLIDXQEWTN4Y656HR54X54AFUUSEFAGJWDEMDB56ZZBKKHT5Q76QC
IKGIGFOCGCV2GITFXC7JB4QMQ4FUMZLIUCU26UGRUOW774IIID5AC
UK4TUMBXYX5SGXO443JQ7I6LDNF62VYGBEWQF2ZFJSLS6H72EMFQC
NTYQUA24YJBD45JOTFY3O4PEA4GUUV34C7MX6ORSJENBFIYACT7QC
LDIYGOU473O3HPMOY2VDF5VTBDVUMJEB23V427KAEFGD64JGV4BAC
D5KRDRYYRE577UK2HETVORW7HQQUI7ZGLDG6N56QLDDOX6GP4O5QC
FS2ITYYHBLFT66YUC3ENPFYI2HOYHOVEPQIN7NQR6KF5MEK4NKZAC
D4FEFHQCSILZFQ5VLWNXAIRZNUMCDNGJSM4UJ6T6FDMMIWYRYILQC
json = require 'json'
function main(args)
local infile = io.open(args[1])
local manifest_s = infile:read('*a')
infile:close()
local manifest = json.decode(manifest_s)
local core_filenames = {}
for k,v in pairs(manifest) do
if not starts_with(k, 'fw_') then
table.insert(core_filenames, k)
end
end
table.sort(core_filenames)
for _,core in ipairs(core_filenames) do
local filename = ('%04d'):format(manifest[core])..'-'..core
local f = io.open(filename)
if f then
print(f:read('*a'))
print('')
end
end
end
function starts_with(s, prefix)
if #s < #prefix then
return false
end
for i=1,#prefix do
if s:sub(i,i) ~= prefix:sub(i,i) then
return false
end
end
return true
end
main(arg)
# The on-disk representation of freewheeling apps
When you start up a freewheeling app, you'll see a directory printed out in
the parent terminal (always launch it from a terminal window):
```
new edits will go to /home/...
```
When editing such an app using the driver (see [README.md](README.md)), new
definitions will go into this directory. Let's call it `$SAVE_DIR` in the rest
of this doc.
It is always safe to move such definitions into this repo. (We'll call it `.`
in the rest of this doc.) You'll want to do this if you're sharing them with
others, and it's also helpful if the driver crashes on your app. Moving
definitions will never, ever change app behavior.
```sh
$ mv -i $SAVE_DIR/[0-9]* . # should never clobber any existing files
$ mv $SAVE_DIR/head . # expected to clobber the existing file
```
Try looking inside the `head` file with a text editor. It'll contain a number,
the current version of the _manifest_ for this app. For example:
```
478
```
If you moved the files you should see such a file in `.`. If you open this
file, you'll see a JSON table from definition names to version ids. For
example:
```
{ "a": 273, "b": 478}
```
This means the current definition of `a` is in `0273-a` and of `b` in
`0478-b`.
Poking around these files gets repetitive, so there's a tool to streamline
things:
```
```
`stitch-live.lua` takes a manifest file as its argument, and prints out all
the definitions that make up the app at that version.
To compare two versions of the app, use `stitch-live.lua` to copy the
definitions in each into a separate file, and use a file comparison tool (e.g.
`diff`) to compare the two files.
# Scenarios considered in designing this representation
* Capture history of changes.
- Though it is perhaps too fine-grained and noisy.
* Merge changes from non-live forks to live ones.
- New files in repo can't hide changes in save dir, because filenames are
always disjoint between the two.
- This doesn't apply yet to live updates. Two forks of a single live app
will likely have unnecessary merge conflicts.
* No special tools required to publish changes to others.
- Just move files from save dir to repo.
# Scenarios I _would_ like to take into consideration in the future
* Cleaner commits; it's clear what changed.
* merge changes between live forks.
lua tools/stitch-live.lua 0478-fwmanifest
This means the current state of the app is in a file called `0478-fwmanifest`.
-- on incoming messages to a specific file, however, the app must:
-- save the message's value to a new, smallest unused numeric prefix
-- execute the value
-- if there's an error, go back to the previous value of the same
-- definition if one exists
-- on incoming messages to a specific file, the app must:
-- determine the definition name from the first word
-- execute the value, returning any errors
-- look up the filename for the definition or define a new filename for it
-- save the message's value to the filename
Live.head = 0
Live.next_version = 1
Live.history = {} -- array of filename roots corresponding to each numeric prefix
Live.manifest = {} -- mapping from roots to numeric prefixes as of version Live.head
Live.filenames_to_load = {} -- filenames in order of numeric prefix
Live.filename = {} -- map from definition name to filename (including numeric prefix)
Live.final_prefix = 0
local files = {}
live.append_files_with_numeric_prefix('', files)
table.sort(files)
live.check_integrity(files)
live.append_files_with_numeric_prefix(love.filesystem.getSaveDirectory(), files)
table.sort(files)
live.check_integrity(files)
Live.history = live.load_history(files)
Live.next_version = #Live.history + 1
local head_string = love.filesystem.read('head')
Live.head = tonumber(head_string)
if Live.head > 0 then
Live.manifest = json.decode(love.filesystem.read(live.versioned_manifest(Live.head)))
end
live.load_everything_in_manifest()
end
function live.append_files_with_numeric_prefix(dir, files)
for _,file in ipairs(love.filesystem.getDirectoryItems(dir)) do
if file:match('^%d') then
table.insert(files, file)
end
end
end
function live.check_integrity(files)
local manifest_found, file_found = false, false
local expected_index = 1
for _,file in ipairs(files) do
for numeric_prefix, root in file:gmatch('(%d+)-(.+)') do
-- only runs once
local index = tonumber(numeric_prefix)
-- skip files without numeric prefixes
if index ~= nil then
if index < expected_index then
print(index, expected_index)
end
assert(index >= expected_index)
if index > expected_index then
assert(index == expected_index+1)
assert(manifest_found and file_found)
expected_index = index
manifest_found, file_found = false, false
end
assert(index == expected_index)
if root == 'fwmanifest' then
assert(not manifest_found)
manifest_found = true
else
assert(not file_found)
file_found = true
end
-- if necessary, copy files from repo to save dir
if io.open(love.filesystem.getSaveDirectory()..'/0000-freewheeling-start') == nil then
print('copying all definitions from repo to save dir')
for _,filename in ipairs(love.filesystem.getDirectoryItems('')) do
for numeric_prefix, root in filename:gmatch('(%d+)-(.+)') do
-- only runs once
local buf = love.filesystem.read(filename)
print('copying', filename)
love.filesystem.write(filename, buf)
end
function live.load_history(files)
local result = {}
for _,file in ipairs(files) do
for numeric_prefix, root in file:gmatch('(%d+)-(.+)') do
-- load files from save dir
for _,filename in ipairs(love.filesystem.getDirectoryItems('')) do
for numeric_prefix, root in filename:gmatch('(%d+)-(.+)') do
local index = tonumber(numeric_prefix)
-- skip
if index ~= nil then
if root ~= 'fwmanifest' then
assert(index == #result+1)
table.insert(result, root)
end
if tonumber(numeric_prefix) > 0 then -- skip 0000
Live.filename[root] = filename
table.insert(Live.filenames_to_load, filename)
Live.final_prefix = math.max(Live.final_prefix, tonumber(numeric_prefix))
function live.load_everything_in_manifest()
local files_to_load = {}
for k,v in pairs(Live.manifest) do
-- Most keys in the manifest are definitions. If we need to store any
-- metadata we'll do it in keys starting with a specific prefix.
if not starts_with(k, 'fw_') then
local root, index = k, v
local filename = live.versioned_filename(index, root)
table.insert(files_to_load, filename)
end
end
table.sort(files_to_load)
for _,filename in ipairs(files_to_load) do
function live.load_all()
for _,filename in ipairs(Live.filenames_to_load) do
--? print('loading', filename)
Live.manifest[APP] = love.filesystem.getIdentity() -- doesn't need to be persisted, but no harm if it does..
live.send_to_driver(json.encode(Live.manifest))
Live.filename[APP] = love.filesystem.getIdentity()
live.send_to_driver(json.encode(Live.filename))
local definition_name = buf:match('^%S+%s+(%S+)')
Live.manifest[definition_name] = nil
live.eval(definition_name..' = nil') -- ignore errors which will likely be from keywords like `function = nil`
local next_filename = live.versioned_filename(Live.next_version, definition_name)
love.filesystem.write(next_filename, '')
table.insert(Live.history, definition_name)
live.roll_forward()
local definition_name = buf:match('^%s*%S+%s+(%S+)')
if Live.filename[definition_name] then
local index = table.find(Live.filenames_to_load, Live.filename[definition_name])
table.remove(Live.filenames_to_load, index)
live.eval(definition_name..' = nil') -- ignore errors which will likely be from keywords like `function = nil`
love.filesystem.remove(Live.filename[definition_name])
Live.filename[definition_name] = nil
end
local next_filename = live.versioned_filename(Live.next_version, definition_name)
love.filesystem.write(next_filename, buf)
table.insert(Live.history, definition_name)
Live.manifest[definition_name] = Live.next_version
live.roll_forward()
-- eval succeeded without errors; persist the definition
local filename = Live.filename[definition_name]
if filename == nil then
Live.final_prefix = Live.final_prefix+1
filename = ('%04d-%s'):format(Live.final_prefix, definition_name)
table.insert(Live.filenames_to_load, filename)
Live.filename[definition_name] = filename
end
love.filesystem.write(filename, buf)
end
-- update Live.Head and record the new Live.Manifest (which caller has already modified)
function live.roll_forward()
Live.manifest[PARENT] = Live.head
local manifest_filename = live.versioned_manifest(Live.next_version)
love.filesystem.write(manifest_filename, json.encode(Live.manifest))
Live.head = Live.next_version
love.filesystem.write('head', tostring(Live.head))
Live.next_version = Live.next_version + 1
end
-- update app.Head and reload app.Manifest appropriately
function live.roll_back()
Live.head = Live.manifest[PARENT]
love.filesystem.write('head', tostring(Live.head))
local previous_manifest_filename = live.versioned_manifest(Live.head)
Live.manifest = json.decode(love.filesystem.read(previous_manifest_filename))
This file contains no definition, but is used as a marker in the save dir to
indicate all definitions have been copied from the repo to the save dir.