Update docker build, clean up migration for payments tables.

[?]

Mar 2, 2017, 5:28 AM

EW2XN7KUMCAQNVFJJ5YTAVDZCPHNWDOEDMRFBUGLY6IE2HKNNX5AC

Dependencies

[2] 45AI46JN Move readme to inception.md
[3] LEINLS3X Update deployment documentation.
[4] WZFQDWW4 Add retrieval/storage of current exchange rate data to payment recording.
[5] Q5X5RYQL stylish-haskell reformatting
[6] HMDM3B55 Implement core of payments/billing infrastructure.
[7] 64C6AWH6 Rename Ananke -> Quixotic, project reboot.
[8] 7HPY3QPF Fix linting errors. (yay hlint!)
[9] DXIGERDT Change order of Docker build to avoid rebuilding the universe.
[10] 4ZLEDBK7 Initial attempts at dockerizing, cabal isn't cooperating.
[11] AXKKXBWN Initial attempt at writing down my ideas for a company based on trust.
[12] M4KM76DG Merge branch 'stackify'
[13] DFOBMSAO Initial work on payments API
[14] WAIX6AGN Add event serialization for PaymentRequest & Payment
[15] Z7KS5XHH Very WIP. Wow.
[16] O5FVTOM6 Undo JSON silliness, enable a couple more routes.
[17] V2VDN77H Enable postgres configuration via environment variable for Heroku.
[18] EPOYLP7O A little .gitignore cleanup.
[19] E2KOBKIJ Add setup script detailing the setup of the docker host.
[20] QMRKFEPG Refactor QDB to use a free monad algebra instead.
[21] TA6RIVTQ Revising...
[22] Y3LIJ5US Add handler for CreatePaymentRequest
[23] ADMKQQGC Initial empty Snap project.
[24] ZKJJVD2H Fix aftok-server runit script permissions.
[25] RB2ETNIF Add skeletal PureScript client project.
[26] OBFPJS2G Project successfully builds and tests under nix.
[27] POX3UAMT Enabling logging of time to contributor/project accounts
[28] O227CEAV Adds storage of original event JSON for some DBOp constructors.
[29] GCVQD44V Create amends endpoint, switch to UUID primary keys
[30] O722AOKE Add route to allow crediting of events to users/projects.
[31] RPAJLHMT Change to use UUIDs instead of ints for primary keys.
[32] 45QJYWN3 Fixing up the README. Still struggling with the ending.
[33] BSIUHCGF Add payment response handler.
[34] DLZRD7VB Add a preliminary, probably somewhat broken set of setup instructions.
[35] IRG4KNAE Trivial deletion.
[36] SEWTRB6S Implement payment request creation functions.
[37] 2WOOGXDH Use dbmigrations to manage database state.
[38] Z3MK2PJ5 Add GET handler for retrieving auction data.
[39] EQXRXRZD Changed to use tasty instead of test-framework
[40] AL37SVTC Implement payments service endpoints.
[41] 7VGYLTMU Clean up schema version handling.
[42] 2XQD6KKK Add invitation logic and clean up DBProg error handling.
[43] JEOPOOPT Dockerfile now builds correctly.
[44] ASF3UPJL Add auction creation and bid handlers
[45] NLZ3JXLO Fix formatting with stylish-haskell.
[46] 73NDXDEZ Begin implementation of billing event persistence.
[47] RSEB2NFG Replacing Snap with Scotty.
[48] NVOCQVAS Initial failing tests.
[49] PBD7LZYQ Postgres & auth are beginning to function.
[50] RN7EI6IN Update database layer to use CreditTo
[51] NTPC7KJE Trivial changes, feature scratchpad.
[52] KEP5WUFJ Convert project to stack-based build.
[53] QO4NFWIY Added sample config file.
[54] T44T2PDL Rename trust.txt to README.md
[55] LAROLAYU WIP
[56] EZQG2APB Update task list.
[57] ZP62WC47 Begin conversion to build with stack.
[58] 373LXH2X Add MAYBE.md, update task list.
[59] 2Y2QZFVF Switch to more modern cabal2nix-based workflow.
[60] M3KUPGZK Add invitation email template.
[61] 2LZYVHFS Upgrade to Stack-based build in Docker
[62] 5W5M56VJ Move library code to 'lib'
[63] W35DDBFY Factor common JSON conversions up into client lib module.
[64] 4U7F3CPI THE GREAT RENAMING OF THINGS!
[*] SPJCFHXW Update shell scripts to point to https://aftok.com and prompt for input.
[*] VV6ESCEV Update .gitignore for client libs.
[*] IZEVQF62 Work in progress replacing sqlite with postgres.
[*] JFOEOFGA stylish-haskell formatting.
[*] BWN72T44 Don't accept work timestamp from an external source.

Change contents

file deletion: notes.md (----------)

[5.2]→[5.19097:19129](∅→∅),[5.19129]→[5.17955:17955](∅→∅)

Aftok
=====

> *"I have to figure out how to balance the work I can make a living on with the
> work I can’t, because the work I can’t make a living on is more important."*
> -- [Meredith L. Patterson](https://medium.com/message/how-i-explained-heartbleed-to-my-therapist-4c1dbcbe1099)

It's impossible to make a living writing open-source software. Certainly, there
are exceptions; a few companies eke out a relatively spare existence selling
"enterprise support" licenses for free software, and lots of people employed by
private companies contribute to open-source products on time paid for by their
employer. Nonetheless, a great deal of the open-source software that developed
is a labor of love. OSS developers are artists, but they're fortunate that
their art is in a medium that also, at present, makes it easy enough to find
paid work that few of us have to starve while making our art.

Unlike many other arts (excepting music) open-source software development is
frequently a deeply collaborative endeavor.

* Trust cannot be imposed, it can only be grown.
* Last section needs to be more positive, deemphasize risk.

(12:36:54 PM) nuttycom1: Reiterating my earlier question, this time via StackOverflow: http://stackoverflow.com/questions/30466275/http-basic-auth-in-snap
(12:41:03 PM) dmj`: nuttycom1: why not use snap's cookie-based auth?
(12:41:33 PM) carter_cloud: mightybyte: lpsmith did that posgres-simple snaplet transaction bug ever get fixed?
(12:41:40 PM) nuttycom1: dmj`: my clients aren't necessarily browsers.
(12:41:55 PM) nuttycom1: In fact, they're mostly not browsers.
(12:43:17 PM) dmj`: nuttycom1: are they mobile clients?
(12:43:31 PM) nuttycom1: dmj`: no, other servers.
(12:48:21 PM) dmj`: nuttycom1: basic auth isn't secure, you're sending the password (base-64 encoded) on every request, it could be fine if it's all over https. But still, a token based system would be more secure. Are these severs internal / external? (i.e. yours or someone elses)
(12:53:16 PM) nuttycom1: dmj`: Token-based systems are no more secure if they're not over https, but that's a side point; in my experience it's common in any case to use the Authorization headers to carry authentication tokens so that they don't have to pollute the request payload in other ways.
(12:54:14 PM) nuttycom1: I have a bunch of CLI tools that access my services via curl and such, which prompt for auth credentials along the way. Using Basic is convenient for these kinds of tools.
(12:54:49 PM) nuttycom1: All the servers *are* in my control, however. I suppose I could use an X-My-Auth header or some such, but I don't much see the point.
(12:55:38 PM) nuttycom1: At least, they're under my control at the moment, but obviously the API I'm providing is ultimately going to be accessible to anyone on the open web, so I might as well make things easy for them.
(12:59:00 PM) mightybyte: carter_cloud: Yes
(12:59:32 PM) mightybyte: http://blog.melding-monads.com/2015/02/12/announcing-snaplet-postgresql-simple-0-6/
(01:00:42 PM) dmj`: nuttycom1: token based systems only transmit the credentials on the initialize request to generate the token, not on every request, so not sure how it's just as secure. It seems like setting up your own oauth provider would be ideal in this case, especially if you plan on releasing this api to the public. If your servers are not exposed to the outside world, why even have them authenticate against each other, if you're using A
(01:00:43 PM) dmj`: you using a VPC?
(01:00:49 PM) dmj`: initial*
(01:00:52 PM) mightybyte: carter_cloud: I guess that post never hit reddit.
(01:02:17 PM) mightybyte: nuttycom1: I don't know of a basic auth package for snap (probably in part due to the concerns dmj` has pointed out), but I wouldn't expect that it would be too hard to write.
(01:02:42 PM) nuttycom1: mightybyte: Hah.... yeah, I provide one as context in my SO question.
(01:04:49 PM) carter_cloud: mightybyte: btw Stephen Diehl has a neat prototype of a streaming Postgres binding https://github.com/elsen-trading/pgstream
(01:07:49 PM) nuttycom1: dmj`: fair enough; I suppose that I can add token handling to the CLI tools, it's just a little more work since I'll have to encrypt the token before it's stored locally, and require the user to decrypt on each request rather than provide credentials. That's probably the right way to go.
(01:07:51 PM) dmj`: nuttycom1: I can't speak for other token-based systems, but json web tokens are typically encrpyted (HMAC SHA-256), basic auth data is just encoded. So your last line of defense in basic auth is ssl, and your surface area (every request) is much larger. I dunno, if you think basic auth suits your needs then go for it, but it sounds to me like you should become an oauth provider here.
(01:08:16 PM) mightybyte: nuttycom1: responded
(01:09:10 PM) nuttycom1: mightybyte: thanks!
(01:10:45 PM) dmj`: nuttycom1: yea, you can create your own key server! Then create tcp connections from your key server to all other api servers. You should change the key somewhat frequently (This secret key is used to hash all tokens). So your key server can broadcast the new secret to all servers via tcp, then (if you're using haskell and not that node.js single-threaded stuff) you could migrate everyones tokens over to the new key transparen
(01:10:45 PM) dmj`: if new key has been sent, then rehash with it, and put it in the header).
(01:11:16 PM) dmj`: only one-thread, I don't know how people do it
(01:11:20 PM) dmj`: one thread*
(01:12:18 PM) nuttycom1: Interesting, thanks dmj`. Looks like I have some additional investigation to do.
edit in .dockerignore at line 1

[5.1]

[5.2]

.git
.gitignore
.stack-work
edit in .dockerignore at line 5

[5.3]

local
migrations
work

Makefile
Dockerfile
./*.md
./*.cfg
edit in .gitignore at line 19

[66.7]

[67.1]

bin
replacement in Dockerfile at line 14

[5.407]→[5.232:295](∅→∅)

apt-get install -y --no-install-recommends libpq-dev stack

[5.407]

[5.1]

apt-get install -y --no-install-recommends \
build-essential autotools-dev autoconf dh-autoreconf \
libpq-dev libsqlite3-dev \
git stack
replacement in Dockerfile at line 19

[5.2]→[5.2:59](∅→∅)

# Set up /etc/aftok volume for configuration information

[5.2]

[5.59]

# Set up /etc/aftok volume for mounting configuration from the host system
edit in Dockerfile at line 22

[5.102]→[5.102:152](∅→∅)

ADD ./conf/aftok.cfg.example /etc/aftok/aftok.cfg
edit in Dockerfile at line 31

[5.67]

[5.30]

# Install ghc globally so that we don't have to reinstall it
# whenever we change stack.yaml or aftok.cabal
RUN stack --resolver lts-7.16 setup
replacement in Dockerfile at line 37

[5.71]→[5.71:118](∅→∅)

ADD ./docker/stack.yaml /opt/aftok/stack.yaml

[5.71]

[5.440]

ADD ./stack.yaml /opt/aftok/stack.yaml
edit in Dockerfile at line 44

[3.1142]→[3.1142:1211](∅→∅),[5.136]→[5.1024:1025](∅→∅),[5.368]→[5.1024:1025](∅→∅),[5.482]→[5.1024:1025](∅→∅),[5.871]→[5.1024:1025](∅→∅),[3.1211]→[5.1024:1025](∅→∅),[5.1024]→[5.1024:1025](∅→∅)

RUN apt-get install -y libsqlite3-dev
RUN stack install dbmigrations
edit in Dockerfile at line 47

[5.222]→[3.1212:1252](∅→∅)

ADD ./migrations /opt/aftok/migrations
replacement in Makefile at line 2

[3.1272]→[3.1272:1310](∅→∅)

docker build -t aftok/aftok:latest .

[3.1272]

[3.1310]

sudo docker build -t aftok/aftok:latest .
replacement in Makefile at line 4

[3.1311]→[3.1311:1432](∅→∅)

run-local-docker:
docker run --net=host -it -v /home/nuttycom/projects/aftok/docker-conf/:/etc/aftok aftok/aftok:latest

[3.1311]

[3.1432]

run-local-docker: build-container
sudo docker run --net=host -it -v /home/nuttycom/projects/aftok/local/conf/:/etc/aftok aftok/aftok:latest
file addition: bin (d--r------)

[5.2]
file addition: .keep (----------)

[0.703]
file deletion: stack.yaml (----------)

[5.1510]→[5.496:530](∅→∅),[5.530]→[5.319:319](∅→∅)

flags: {}
packages:
- '.'
extra-deps:
- snaplet-postgresql-simple-0.6.0.4
- resource-pool-catchio-0.2.1.0
resolver: lts-5.3
#allow-newer: true
local-bin-path: /opt/aftok/bin
file addition: docs (d--r------)

[5.2]
file move: README.md (----------) → design.md (----------)

[0.754]

[5.1]
file move: features.md (----------) → features.md (----------)

[0.754]

[5.17292]
file move: inception.md (----------) → inception.md (----------)

[0.754]

[5.4]
file move: MAYBE.md (----------) → maybe.md (----------)

[0.754]

[5.1]
file move: TASKS.md (----------) → tasks.md (----------)

[0.754]

[5.1]
replacement in lib/Aftok/Database/PostgreSQL.hs at line 218

[5.264]→[5.392:434](∅→∅),[5.434]→[5.533:623](∅→∅)

storeEvent (CreateSubscription uid bid) =
Just $ storeEventJSON (Just uid) "create_subscription" (createSubscriptionJSON uid bid)

[5.264]

[5.384]

storeEvent (CreateSubscription uid bid t) =
Just $ storeEventJSON (Just uid) "create_subscription" (createSubscriptionJSON uid bid t)
replacement in lib/Aftok/Database/PostgreSQL.hs at line 438

[5.12002]→[5.403:522](∅→∅)

\(project_id, event_id, name, description, recurrence_type, recurrence_count, billing_amount, grace_period_days) \

[5.12002]

[5.12120]

\( project_id, event_id, name, description \
\, recurrence_type, recurrence_count \
\, billing_amount, grace_period_days) \
replacement in lib/Aftok/Database/PostgreSQL.hs at line 460

[5.3943]→[5.518:566](∅→∅)

pgEval dbop @ (CreateSubscription uid bid) = do

[5.3943]

[5.576]

pgEval dbop @ (CreateSubscription uid bid start_date) = do
replacement in lib/Aftok/Database/PostgreSQL.hs at line 465

[5.710]→[5.710:746](∅→∅)

\VALUES (?, ?, ?) RETURNING id"

[5.710]

[5.567]

\VALUES (?, ?, ?, ?) RETURNING id"
edit in lib/Aftok/Database/PostgreSQL.hs at line 469

[5.647]

[5.647]

, fromThyme start_date
edit in lib/Aftok/Database.hs at line 14

[69.1893]

[68.6576]

import Data.Thyme.Time as T (Day)
replacement in lib/Aftok/Database.hs at line 56

[5.13383]→[5.1644:1712](∅→∅)

CreateSubscription :: UserId -> BillableId -> DBOp SubscriptionId

[5.13383]

[5.1712]

CreateSubscription :: UserId -> BillableId -> T.Day -> DBOp SubscriptionId
replacement in lib/Aftok/Json.hs at line 9

[5.156]→[5.918:949](∅→∅)

import ClassyPrelude

[5.156]

[5.1965]

import ClassyPrelude hiding (Day)
edit in lib/Aftok/Json.hs at line 26

[70.58]

[5.3094]

import Data.Thyme.Time (Day)
replacement in lib/Aftok/Json.hs at line 105

[5.2437]→[5.2437:2523](∅→∅)

idJSON :: forall a. Lens' a UUID -> a -> Value
idJSON l a = toJSON . tshow $ view l a

[5.2437]

[5.485]

idValue :: forall a. Lens' a UUID -> a -> Value
idValue l a = toJSON . tshow $ view l a

idJSON :: forall a. Text -> Lens' a UUID -> a -> Value
idJSON t l a = v1 $ obj [ t .= idValue l a ]
replacement in lib/Aftok/Json.hs at line 113

[5.6666]→[5.1197:1247](∅→∅)

obj [ "projectId" .= tshow (pid ^. _ProjectId)

[5.6666]

[5.1247]

obj [ "projectId" .= idValue _ProjectId pid
replacement in lib/Aftok/Json.hs at line 118

[5.286]→[5.286:311](∅→∅),[5.311]→[5.1297:1348](∅→∅)

projectIdJSON pid = v1 $
obj [ "projectId" .= tshow (pid ^. _ProjectId) ]

[5.286]

[5.1693]

projectIdJSON = idJSON "projectId" _ProjectId
replacement in lib/Aftok/Json.hs at line 128

[5.402]→[5.402:427](∅→∅),[5.427]→[5.1517:1568](∅→∅)

auctionIdJSON pid = v1 $
obj [ "auctionId" .= tshow (pid ^. _AuctionId) ]

[5.402]

[5.481]

auctionIdJSON = idJSON "auctionId" _AuctionId
replacement in lib/Aftok/Json.hs at line 132

[5.592]→[5.1569:1694](∅→∅)

obj [ "projectId" .= tshow (x ^. (A.projectId._ProjectId))
, "initiator" .= tshow (x ^. (A.initiator._UserId))

[5.592]

[5.5070]

obj [ "projectId" .= idValue (A.projectId._ProjectId) x
, "initiator" .= idValue (A.initiator._UserId) x
replacement in lib/Aftok/Json.hs at line 143

[5.1894]→[5.1894:1989](∅→∅)

creditToJSON (CreditToUser uid) = v2 $ obj [ "creditToUser" .= tshow (uid ^. _UserId) ]

[5.1894]

[5.1989]

creditToJSON (CreditToUser uid) = v2 $ obj [ "creditToUser" .= idValue _UserId uid ]
replacement in lib/Aftok/Json.hs at line 163

[5.2094]→[5.5450:5483](∅→∅),[5.5483]→[5.2202:2235](∅→∅),[5.2235]→[5.1798:1799](∅→∅),[5.5519]→[5.1798:1799](∅→∅),[5.2133]→[5.1798:1799](∅→∅)

eventIdJSON (EventId eid) = v1 $
obj [ "eventId" .= tshow eid ]

[5.2094]

[5.130]

eventIdJSON = idJSON "eventId" _EventId
replacement in lib/Aftok/Json.hs at line 176

[5.5600]→[5.5600:5641](∅→∅),[5.5641]→[5.2343:2380](∅→∅)

amendmentIdJSON (AmendmentId aid) = v1 $
obj [ "amendmentId" .= tshow aid ]

[5.5600]

[5.1568]

amendmentIdJSON = idJSON "amendmentId" _AmendmentId
replacement in lib/Aftok/Json.hs at line 183

[5.2627]→[5.2627:2691](∅→∅)

[ "projectId" .= (b ^. (B.project . _ProjectId . to tshow))

[5.2627]

[5.2691]

[ "projectId" .= idValue (B.project . _ProjectId) b
replacement in lib/Aftok/Json.hs at line 199

[5.5682]→[5.2998:3188](∅→∅)

createSubscriptionJSON :: UserId -> B.BillableId -> Value
createSubscriptionJSON uid bid = v1 $
obj [ "user_id" .= idJSON _UserId uid
, "billable_id" .= idJSON B._BillableId bid

[5.5682]

[5.621]

createSubscriptionJSON :: UserId -> B.BillableId -> Day -> Value
createSubscriptionJSON uid bid d = v1 $
obj [ "user_id" .= idValue _UserId uid
, "billable_id" .= idValue B._BillableId bid
, "start_date" .= showGregorian d
replacement in lib/Aftok/Json.hs at line 211

[5.3359]→[5.3359:3475](∅→∅)

[ "user_id" .= idJSON (B.customer . _UserId) sub
, "billable_id" .= idJSON (B.billable . B._BillableId) sub

[5.3359]

[5.3475]

[ "user_id" .= idValue (B.customer . _UserId) sub
, "billable_id" .= idValue (B.billable . B._BillableId) sub
replacement in lib/Aftok/Json.hs at line 222

[5.3694]→[4.396:601](∅→∅)

[ "subscription_id" .=
view (subscription . B._SubscriptionId . to tshow) r
, "payment_request_protobuf_64" .=
view (paymentRequest . to (decodeUtf8 . B64.encode . runPut . encodeMessage)) r

[5.3694]

[4.601]

[ "subscription_id" .= idValue (subscription . B._SubscriptionId) r
, "payment_request_protobuf_64" .= view prBytes r
edit in lib/Aftok/Json.hs at line 228

[5.4011]

[5.4011]

where
prBytes = (paymentRequest . to (decodeUtf8 . B64.encode . runPut . encodeMessage))
replacement in lib/Aftok/Json.hs at line 246

[5.1029]→[5.1029:1216](∅→∅)

obj [ "payment_request_id" .= (r ^. (request . _PaymentRequestId . to tshow))
, "payment_protobuf_64" .= (r ^. (payment . to (decodeUtf8 . B64.encode . runPut . encodeMessage)))

[5.1029]

[5.1216]

obj [ "payment_request_id" .= idValue (request . _PaymentRequestId) r
, "payment_protobuf_64" .= view paymentBytes r
edit in lib/Aftok/Json.hs at line 250

[5.2430]

[5.2430]

where
paymentBytes = payment . to (decodeUtf8 . B64.encode . runPut . encodeMessage)
edit in lib/Aftok/Json.hs at line 337

[5.4874]

[5.987]

parseBillable :: Value -> Parser B.Billable
parseBillable = unversion "Billable" p where
--p (Version 1 0) o =

p v o = badVersion "Billable" v o
edit in lib/Aftok/Json.hs at line 344

[5.988]
replacement in migrations/2016-12-31_03-45-17_create-payments.txt at line 58

[5.9257]→[5.9257:9289](∅→∅)

request_data bytea not null

[5.9257]

[5.9289]

request_data bytea not null,
url_key text not null,
request_time timestamp with time zone not null,
billing_date date not null
replacement in migrations/2016-12-31_03-45-17_create-payments.txt at line 68

[5.9513]→[5.9513:9545](∅→∅)

payment_data bytea not null

[5.9513]

[5.9545]

payment_data bytea not null,
payment_date timestamp with time zone not null,
exchange_rates json
edit in migrations/2016-12-31_03-45-17_create-payments.txt at line 78

[5.9667]

[5.9667]

drop type recurrence_t;
edit in migrations/2016-12-31_03-45-17_create-payments.txt at line 80

[5.9694]

[5.9694]

drop type aftok_event_t;
edit in migrations/2016-12-31_03-45-17_create-payments.txt at line 83

[5.9696]
replacement in stack.yaml at line 5

[5.10554]→[5.10554:10594](∅→∅)

git: git@github.com:aftok/bippy.git

[5.10554]

[5.13697]

git: https://github.com/aftok/bippy.git
edit in stack.yaml at line 15

[5.23448]

local-bin-path: ./bin