Attempting to improve JSON handling.
[?]
Apr 17, 2015, 9:44 PM
A6HKMINBNGQLLX4QJMYWKQ4JAEHVJ4HIRVDKPPDI3FJUO2AAB7OQCDependencies
- [2]
XTBSG4C7Adding serveJSON combinator to eliminate some boilerplate from handlers. - [3]
5XFJNUAZStart of addition of project infrastructure. - [4]
NVOCQVASInitial failing tests. - [5]
TNR3TEHKSwitch to Postgres + snaplet arch compiles. - [6]
I2KHGVD4Require project permissions for access to most data. - [7]
TLQ72DSJLenses, sqlite-simple - [8]
EQXRXRZDChanged to use tasty instead of test-framework - [9]
N4NDAZYTInitial implementation of payouts. - [10]
P6NR2CGXBeginning of implementation of depreciation. - [11]
BROSTG5KBeginning of modularization of server. - [12]
EZQG2APBUpdate task list. - [13]
2Y2QZFVFSwitch to more modern cabal2nix-based workflow. - [14]
7XN3I3QJAdd 'loggedIntervals' endpoint. - [15]
GKGVYBZGAdded JSON serialization to TimeLog - [16]
Y35QCWYWMinor improvement in WorkIndex type to eliminate duplicated information. - [17]
64C6AWH6Rename Ananke -> Quixotic, project reboot. - [18]
LD4GLVSFMore database stuff. - [19]
HE3JTXO3Added client call to payouts. - [20]
IZEVQF62Work in progress replacing sqlite with postgres. - [21]
EMVTF2IWWIP moving back to snap. - [22]
VJPT6HDRFix remaining type errors after addition of login handler. - [23]
OBFPJS2GProject successfully builds and tests under nix. - [24]
7DBNV3GVInitial, stack-based impl of time log event reduction. - [25]
W35DDBFYFactor common JSON conversions up into client lib module. - [26]
SLL7262CMake depreciation functions more flexible. - [27]
Z7KS5XHHVery WIP. Wow. - [28]
4IQVQL4TAdded client for payouts endpoint. - [29]
4QX5E5ACInitial compilation of payouts function succeeds. - [30]
2G3GNDDUEvent logging is now functioning in postgres. - [31]
PBD7LZYQPostgres & auth are beginning to function. - [*]
ADMKQQGCInitial empty Snap project.
Change contents
- replacement in TASKS.md at line 11
Design Goals:Design Guidelines: - replacement in TASKS.md at line 20
Library:Required for launch===================Library------- - replacement in TASKS.md at line 31
the invitation* Timeline* Amend Event* Amend operations targeting events older than <commit_delay hours> fail.* MAYBE garnish/reimburse based approach?* Secure the transaction log via inclusion of periodic hashes of the loginto the public blockchain?the invitation + script# * Timeline# * Amend Event# * Amend operations targeting events older than <commit_delay hours> fail. - edit in TASKS.md at line 37
* Add public keys that can be used to sign requests. How does this interactwith certificate-based auth from browsers? Require openpgpjs? - edit in TASKS.md at line 43
* History of payouts (read from blockchain?) - replacement in TASKS.md at line 55
Webserver:* Login* Evaluate OpenID options* Companion Creation* Require user to provide the PGP public key that will be used to authenticate requests* Authentication* Require bodies of allWebserver--------- - replacement in TASKS.md at line 60
Payouts Service:Payouts Service--------------- - edit in TASKS.md at line 76[3.3394]
Future Work===========Library-------* Timeline* Amend Event* MAYBE garnish/reimburse based approach?* Secure the transaction log via inclusion of periodic hashes of the loginto the public blockchain?* User* Add public keys that can be used to sign requests. How does this interactwith certificate-based auth from browsers? Require openpgpjs?* Payouts* History of payouts (read from blockchain?)Webserver---------* Login* Evaluate OpenID options* Companion Creation* Require user to provide the PGP public key that will be used to authenticate requests* Authentication* Require bodies of all requests to be PGP-signed; this will take the place ofother authentication.Payouts Service--------------- - edit in lib/Quixotic/Client.hs at line 12
import Quixotic.Json - replacement in lib/Quixotic/Client.hs at line 26
pure $ payoutsResponse ^. (responseBody . _PayoutsJ)pure $ payoutsResponse ^. responseBody - edit in lib/Quixotic/Database/PostgreSQL.hs at line 21
type QDBM = ReaderT Connection IO - replacement in lib/Quixotic/Database/PostgreSQL.hs at line 115
recordEvent' :: ProjectId -> UserId -> LogEntry -> ReaderT Connection IO ()recordEvent' (ProjectId pid) (UserId uid) (LogEntry a e) = dopquery :: (ToRow d, FromRow r) => Query -> d -> QDBM [r]pquery q d = do - replacement in lib/Quixotic/Database/PostgreSQL.hs at line 118
void . lift $ execute conn"INSERT INTO work_events (project_id, user_id, btc_addr, event_type, event_time, event_meta) \\VALUES (?, ?, ?, ?, ?, ?)"lift $ query conn q dpexec :: (ToRow d) => Query -> d -> QDBM Int64pexec q d = doconn <- asklift $ execute conn q drecordEvent' :: ProjectId -> UserId -> LogEntry -> QDBM EventIdrecordEvent' (ProjectId pid) (UserId uid) (LogEntry a e) = doeventIds <- pquery"INSERT INTO work_events (project_id, user_id, btc_addr, event_type, event_time, event_metadata) \\VALUES (?, ?, ?, ?, ?, ?) \\RETURNING id" - edit in lib/Quixotic/Database/PostgreSQL.hs at line 137
pure . EventId . fromOnly $ DL.head eventIds - replacement in lib/Quixotic/Database/PostgreSQL.hs at line 139
readWorkIndex' :: ProjectId -> ReaderT Connection IO WorkIndexamendEvent' :: EventId -> LogModification -> QDBM ()amendEvent' (EventId eid) (TimeChange mt t) =void $ pexec"INSERT INTO event_time_amendments (event_id, mod_time, event_time) VALUES (?, ?, ?)"( eid, mt ^. _ModTime, t )amendEvent' (EventId eid) (AddressChange mt addr) =void $ pexec"INSERT INTO event_addr_amendments (event_id, mod_time, btc_addr) VALUES (?, ?, ?)"( eid, mt ^. _ModTime, addr ^. _BtcAddr )amendEvent' (EventId eid) (MetadataChange mt v) =void $ pexec"INSERT INTO event_metadata_amendments (event_id, mod_time, btc_addr) VALUES (?, ?, ?)"( eid, mt ^. _ModTime, v )readWorkIndex' :: ProjectId -> QDBM WorkIndex - replacement in lib/Quixotic/Database/PostgreSQL.hs at line 157
conn <- askrows <- lift $ query connrows <- pquery - replacement in lib/Quixotic/Database/PostgreSQL.hs at line 162
newAuction' :: ProjectId -> Auction -> ReaderT Connection IO AuctionIdnewAuction' :: ProjectId -> Auction -> QDBM AuctionId - replacement in lib/Quixotic/Database/PostgreSQL.hs at line 164
conn <- askaucIds <- lift $ query connaucIds <- pquery - replacement in lib/Quixotic/Database/PostgreSQL.hs at line 169
readAuction' :: AuctionId -> ReaderT Connection IO (Maybe Auction)readAuction' :: AuctionId -> QDBM (Maybe Auction) - replacement in lib/Quixotic/Database/PostgreSQL.hs at line 171
conn <- askrows <- lift $ query connrows <- pquery - replacement in lib/Quixotic/Database/PostgreSQL.hs at line 176
recordBid' :: AuctionId -> Bid -> ReaderT Connection IO ()recordBid' :: AuctionId -> Bid -> QDBM () - replacement in lib/Quixotic/Database/PostgreSQL.hs at line 178
conn <- askvoid . lift $ execute connvoid $ pexec - replacement in lib/Quixotic/Database/PostgreSQL.hs at line 187
readBids' :: AuctionId -> ReaderT Connection IO [Bid]readBids' :: AuctionId -> QDBM [Bid] - replacement in lib/Quixotic/Database/PostgreSQL.hs at line 189
conn <- askrows <- lift $ query connrows <- pquery - replacement in lib/Quixotic/Database/PostgreSQL.hs at line 194
createUser' :: User -> ReaderT Connection IO UserIdcreateUser' :: User -> QDBM UserId - replacement in lib/Quixotic/Database/PostgreSQL.hs at line 196
conn <- askuids <- lift $ query connuids <- pquery - replacement in lib/Quixotic/Database/PostgreSQL.hs at line 201
findUser' :: UserId -> ReaderT Connection IO (Maybe User)findUser' :: UserId -> QDBM (Maybe User) - replacement in lib/Quixotic/Database/PostgreSQL.hs at line 203
conn <- askusers <- lift $ query connusers <- pquery - replacement in lib/Quixotic/Database/PostgreSQL.hs at line 208
findUserByUserName' :: UserName -> ReaderT Connection IO (Maybe QDBUser)findUserByUserName' :: UserName -> QDBM (Maybe QDBUser) - replacement in lib/Quixotic/Database/PostgreSQL.hs at line 210
conn <- askusers <- lift $ query connusers <- pquery - replacement in lib/Quixotic/Database/PostgreSQL.hs at line 215
createProject' :: Project -> ReaderT Connection IO ProjectIdcreateProject' :: Project -> QDBM ProjectId - replacement in lib/Quixotic/Database/PostgreSQL.hs at line 218
conn <- askpids <- lift $ query connpids <- pquery - replacement in lib/Quixotic/Database/PostgreSQL.hs at line 222
void . lift $ execute connvoid $ pexec - replacement in lib/Quixotic/Database/PostgreSQL.hs at line 227
findUserProjects' :: UserId -> ReaderT Connection IO [QDBProject]findUserProjects' :: UserId -> QDBM [QDBProject] - replacement in lib/Quixotic/Database/PostgreSQL.hs at line 229
conn <- askresults <- lift $ query connresults <- pquery - replacement in lib/Quixotic/Database/PostgreSQL.hs at line 236
postgresQDB :: QDB (ReaderT Connection IO)postgresQDB :: QDB QDBM - edit in lib/Quixotic/Database/PostgreSQL.hs at line 239
, amendEvent = amendEvent' - replacement in lib/Quixotic/Database.hs at line 25
{ recordEvent :: ProjectId -> UserId -> LogEntry -> m (){ recordEvent :: ProjectId -> UserId -> LogEntry -> m EventId, amendEvent :: EventId -> LogModification -> m () - edit in lib/Quixotic/Interval.hs at line 5
, intervalJSON, parseIntervalJSON - replacement in lib/Quixotic/Interval.hs at line 10
import Control.Lensimport Control.Lens(makeLenses, (^.))import Data.Aesonimport Data.Aeson.Types - edit in lib/Quixotic/Interval.hs at line 27[3.121]
intervalJSON :: Interval -> ValueintervalJSON ival = object ["start" .= (ival ^. start), "end" .= (ival ^. end)]parseIntervalJSON :: Value -> Parser IntervalparseIntervalJSON (Object v) = interval <$> v .: "start" <*> v .: "end"parseIntervalJSON _ = mzero - edit in lib/Quixotic/Json.hs at line 2
{-# LANGUAGE DeriveDataTypeable #-} - replacement in lib/Quixotic/Json.hs at line 9
import Data.Aesonimport qualified Data.Map as Mimport Data.Aesonimport Data.Aeson.Typesimport Data.Dataimport qualified Data.Attoparsec.ByteString.Char8 as Pimport qualified Data.ByteString.Char8 as C - edit in lib/Quixotic/Json.hs at line 16
import Quixotic.Intervalimport Quixotic.TimeLog - replacement in lib/Quixotic/Json.hs at line 17
newtype PayoutsJ = PayoutsJ PayoutsmakePrisms ''PayoutsJimport qualified Language.Haskell.TH as THimport Language.Haskell.TH.Quote - replacement in lib/Quixotic/Json.hs at line 20
instance ToJSON PayoutsJ wheretoJSON (PayoutsJ p) =toJSON $ M.mapKeys (^. _BtcAddr) pdata Version = Version { majorVersion :: Word8, minorVersion :: Word8, trivialVersion :: Word8} deriving (Typeable, Data) - replacement in lib/Quixotic/Json.hs at line 25[3.485]→[3.2144:2177](∅→∅),[3.2177]→[3.731:748](∅→∅),[3.731]→[3.731:748](∅→∅),[3.748]→[2.77:126](∅→∅)
instance FromJSON PayoutsJ whereparseJSON v =PayoutsJ . M.mapKeys BtcAddr <$> parseJSON vprintVersion :: Version -> TextprintVersion Version{..} = intercalate "." (fmap tshow [majorVersion, minorVersion, trivialVersion]) - replacement in lib/Quixotic/Json.hs at line 28
newtype IntervalJ = IntervalJ IntervalmakePrisms ''IntervalJversionParser :: P.Parser VersionversionParser = Version <$> P.decimal <*> (P.char '.' >> P.decimal) <*> (P.char '.' >> P.decimal)versioned :: Version -> Value -> Valueversioned ver v = object [ "schemaVersion" .= printVersion ver, "value" .= v ] - replacement in lib/Quixotic/Json.hs at line 35
instance ToJSON IntervalJ wheretoJSON (IntervalJ ival) =object ["start" .= (ival ^. start), "end" .= (ival ^. end)]version :: QuasiQuoterversion = QuasiQuoter { quoteExp = quoteVersionExp, quotePat = error "Pattern quasiquotation of versions not supported.", quoteType = error "Type quasiquotation of versions not supported.", quoteDec = error "Dec quasiquotation of versions not supported."} - edit in lib/Quixotic/Json.hs at line 42
instance FromJSON IntervalJ whereparseJSON (Object v) =fmap IntervalJ $ interval <$> v .: "start" <*> v .: "end"parseJSON _ = mzero - replacement in lib/Quixotic/Json.hs at line 43
newtype ProjectJ = ProjectJ ProjectmakePrisms ''ProjectJquoteVersionExp :: String -> TH.Q TH.ExpquoteVersionExp s = dov <- either (fail . show) pure $ P.parseOnly versionParser (C.pack s)dataToExpQ (const Nothing) v - replacement in lib/Quixotic/Json.hs at line 48
instance ToJSON ProjectJ wheretoJSON (ProjectJ p) =object [ "projectName" .= (p ^. projectName), "inceptionDate" .= (p ^. inceptionDate), "initiator" .= (p ^. (initiator._UserId)) ]unversion :: (Version -> Value -> Parser a) -> Value -> Parser aunversion f (Object v) = dovers <- v .: "schemaVersion"vers' <- either (\_ -> mzero) pure $ P.parseOnly versionParser (encodeUtf8 vers)value <- v .: "value"f vers' valueunversion _ _ = mzero - replacement in lib/Quixotic/Json.hs at line 56
newtype WidxJ = WidxJ WorkIndexmakePrisms ''WidxJprojectJSON :: Project -> ValueprojectJSON p =object [ "projectName" .= (p ^. projectName), "inceptionDate" .= (p ^. inceptionDate), "initiator" .= (p ^. (initiator._UserId)) ] - edit in lib/Quixotic/Json.hs at line 62
instance ToJSON WidxJ wheretoJSON (WidxJ widx) =toJSON $ (fmap IntervalJ) <$> (M.mapKeysWith (++) (^._BtcAddr) widx) - replacement in lib/Quixotic/TimeLog.hs at line 12
, workIndex, workIndex, workIndexJSON - edit in lib/Quixotic/TimeLog.hs at line 14
, EventId(EventId), _EventId, eventIdJSON, ModTime(ModTime), _ModTime, LogModification(..) - replacement in lib/Quixotic/TimeLog.hs at line 18
, Payouts, Payouts(..), _Payouts - edit in lib/Quixotic/TimeLog.hs at line 27
import Data.Aeson.Types - edit in lib/Quixotic/TimeLog.hs at line 34
import Quixotic.Json - edit in lib/Quixotic/TimeLog.hs at line 63
newtype EventId = EventId Int64 deriving (Show, Eq)makePrisms ''EventIdnewtype ModTime = ModTime UTCTimemakePrisms ''ModTimedata LogModification = TimeChange ModTime UTCTime| AddressChange ModTime BtcAddr| MetadataChange ModTime A.Value - replacement in lib/Quixotic/TimeLog.hs at line 78
type Payouts = Map BtcAddr Rationalnewtype Payouts = Payouts (Map BtcAddr Rational)makePrisms ''PayoutspayoutsJSON :: Payouts -> ValuepayoutsJSON (Payouts m) = toJSON $ MS.mapKeys (^. _BtcAddr) mparsePayoutsJSON :: Value -> Parser PayoutsparsePayoutsJSON v =Payouts . MS.mapKeys BtcAddr <$> parseJSON vinstance A.ToJSON Payouts wheretoJSON = versioned (Version 1 0 0) . payoutsJSONinstance A.FromJSON Payouts whereparseJSON v = let parsePayouts (Version 1 0 0) = parsePayoutsJSONparsePayouts v' = \_ -> fail . show $ printVersion v'in unversion parsePayouts $ v - edit in lib/Quixotic/TimeLog.hs at line 100
workIndexJSON :: WorkIndex -> ValueworkIndexJSON widx = toJSON $ (fmap intervalJSON) <$> (MS.mapKeysWith (++) (^._BtcAddr) widx)eventIdJSON :: EventId -> ValueeventIdJSON (EventId eid) = toJSON eid - replacement in lib/Quixotic/TimeLog.hs at line 123
in fmap (\kt -> toRational $ kt / totalTime) keyTimesin Payouts $ fmap (\kt -> toRational $ kt / totalTime) keyTimes - edit in payouts/Main.hs at line 14
import Quixotic.TimeLog - replacement in payouts/Main.hs at line 59
payouts <- currentPayouts (qcConfig cfg)(Payouts p) <- currentPayouts (qcConfig cfg) - replacement in payouts/Main.hs at line 61
putStrLn (tshow unspent)putStrLn (tshow payouts)[3.2850]putStrLn . tshow $ unspentputStrLn . tshow $ p - edit in quixotic.cabal at line 36
, template-haskell - replacement in server/Main.hs at line 53
addRoutes [ ("login", requireLogin >> (redirect "/home"))addRoutes [ ("login", requireLogin >> (redirect "/home")) - replacement in server/Main.hs at line 55
, ("projects/:projectId/logStart/:btcAddr", method POST $ logWorkHandler StartWork), ("projects/:projectId/logEnd/:btcAddr", method POST $ logWorkHandler StopWork), ("projects/:projectId/log/:btcAddr", serveJSON WidxJ $ method GET loggedIntervalsHandler), ("projects/:projectId", serveJSON ProjectJ $ method GET projectGetHandler), ("projects/:projectId/logStart/:btcAddr", serveJSON eventIdJSON . method POST $ logWorkHandler StartWork), ("projects/:projectId/logEnd/:btcAddr", serveJSON eventIdJSON . method POST $ logWorkHandler StopWork), ("projects/:projectId/log/:btcAddr", serveJSON workIndexJSON $ method GET loggedIntervalsHandler), ("projects/:projectId", serveJSON projectJSON $ method GET projectGetHandler) - replacement in server/Main.hs at line 60
, ("projects", serveJSON (fmap (ProjectJ._project)) $ method GET projectListHandler), ("payouts/:projectId", serveJSON PayoutsJ $ method GET payoutsHandler), ("payouts/:projectId", serveJSON id $ method GET payoutsHandler) - replacement in server/Quixotic/Snaplet/WorkLog.hs at line 20
logWorkHandler :: EventType -> Handler App App ()logWorkHandler :: EventType -> Handler App App EventId