Continue refactor

quickdudley

May 16, 2022, 12:04 AM

NWVUHV6ZKCRTT3UEXCBWPSGX4CGWXGLK2QTDCUN4L2JKL56N2SNQC

Dependencies

• [2] JZRK6Q4K Refactor (preparation for multiple models)
• [3] SJHJS463 Model trainer: main function to open sdl window and launch second thread for actual training
• [4] ROQCAPZJ Begin function for showing the map (for now just opens SDL window)
• [5] CWOSQTC4 Trust tensorflow's thread safety (it wasn't the cause of an earlier bug)
• [6] 6AXPZL5P Try to offload tight loop to rust (for now it segfaults)
• [7] VKA5CCGC Add pysdl2 to the trainmodel dependencies so I'll be able to visualize what's going on.
• [8] E742MTJA Fix segfault (don't pass pointers between functions in different SDL versions), minor refactor
• [9] QBDHX7BH Add a makefile to express python dependencies since pipenv doesn't like tensorflow
• [10] 3QGP6RXL Automatically format python files
• [11] ZLSXVDET Model trainer: export model in tfjs format once trained
• [12] 7ML3OFE7 Model trainer: initial train and visualize thread pair (total crust mass; todo: altitude instead)
• [13] EB3DTD43 Model trainer: use ground altitude instead of crust mass distribution
• [14] 2ABZP2KN Model trainer: save map.png after training
• [15] 6W7MFV2F D20-based neural network architecture
• [*] ZM2EMAZO Start doing python multi-module stuff properly

Change contents

• replacement in trainmodel/src/model.py at line 5
  [3.22]→[3.2142:2143](∅→∅)
  −
  [3.22]

  [3.0]
  + import os
• replacement in trainmodel/src/model.py at line 9
  [3.2178]→[3.2178:2255](∅→∅)
  − activation=keras.activations.softplus)(inputs)
  [3.2178]

  [3.2255]
  + activation=tf.math.asinh)(inputs)
• replacement in trainmodel/src/model.py at line 11
  [3.2289]→[3.2289:2440](∅→∅)
  − activation=keras.activations.softplus)(layer1)

  − layer3 = keras.layers.Dense(20, activation=keras.activations.softplus)(
  [3.2289]

  [3.2440]
  + activation=tf.math.asinh)(layer1)

  + layer3 = keras.layers.Dense(20, activation=tf.math.asinh)(
• replacement in trainmodel/src/model.py at line 14
  [3.2490]→[3.2490:2564](∅→∅)
  − layer4 = keras.layers.Dense(20, activation=keras.activations.softplus)(
  [3.2490]

  [3.2564]
  + layer4 = keras.layers.Dense(20, activation=tf.math.asinh)(
• replacement in trainmodel/src/model.py at line 22
  [3.2982]→[3.2982:3056](∅→∅)
  − layer8 = keras.layers.Dense(20, activation=keras.activations.softplus)(
  [3.2982]

  [3.3056]
  + layer8 = keras.layers.Dense(20, activation=tf.math.asinh)(
• replacement in trainmodel/src/model.py at line 24
  [3.3114]→[3.3114:3188](∅→∅)
  − layer9 = keras.layers.Dense(20, activation=keras.activations.softplus)(
  [3.3114]

  [3.3188]
  + layer9 = keras.layers.Dense(20, activation=tf.math.asinh)(
• replacement in trainmodel/src/model.py at line 26
  [3.3246]→[3.3246:3321](∅→∅)
  − layer10 = keras.layers.Dense(20, activation=keras.activations.softplus)(
  [3.3246]

  [3.3321]
  + layer10 = keras.layers.Dense(20, activation=tf.math.asinh)(
• replacement in trainmodel/src/model.py at line 28
  [3.3379]→[3.3379:3454](∅→∅)
  − layer11 = keras.layers.Dense(20, activation=keras.activations.softplus)(
  [3.3379]

  [3.3454]
  + layer11 = keras.layers.Dense(20, activation=tf.math.asinh)(
• replacement in trainmodel/src/model.py at line 30
  [3.3521]→[3.3521:3596](∅→∅)
  − layer12 = keras.layers.Dense(20, activation=keras.activations.softplus)(
  [3.3521]

  [3.3596]
  + layer12 = keras.layers.Dense(20, activation=tf.math.asinh)(
• replacement in trainmodel/src/model.py at line 35
  [3.3728]→[3.1195:1208](∅→∅),[3.1195]→[3.1195:1208](∅→∅)
  − def model():
  [3.3728]

  [3.3729]
  + def model(chans = 1):
• replacement in trainmodel/src/model.py at line 39
  [3.3819]→[3.3819:3934](∅→∅)
  − outputs = keras.layers.Dense(1, activation=keras.activations.sigmoid)(

  − keras.layers.concatenate([d1, d2]))
  [3.3819]

  [3.3934]
  + m = keras.layers.Dense(20, activation=tf.math.asinh)(

  + keras.layers.concatenate([d1,d2]))

  + outputs = keras.layers.Dense(chans, activation=keras.activations.sigmoid)(m)
• replacement in trainmodel/src/model.py at line 62
  [2.41]→[2.41:180](∅→∅)
  − self.heightmap = model()

  − self.heightmap.compile(optimizer=keras.optimizers.RMSprop(),

  − loss=shore_focused_loss)
  [2.41]
  + self.members = {}

  +

  + def heightmap(self):

  + if 'heightmap' not in self.members:

  + self.members['heightmap'] = model()

  + self.members['heightmap'].compile(

  + optimizer=keras.optimizers.RMSprop(),

  + loss=shore_focused_loss)

  + return self.members['heightmap']

  +

  + def save(self, path):

  + path = os.fspath(path)

  + try:

  + os.makedirs(path)

  + except FileExistsError:

  + None

  + for name, model in self.members.items():

  + model.save(f'{path}{os.sep}{name}')

  +
• replacement in trainmodel/src/drawmap.py at line 28
  [3.5631]→[2.181:217](∅→∅)
  − outputs = m.heightmap(inputs)
  [3.5631]

  [3.5657]
  + outputs = m.heightmap()(inputs)
• edit in trainmodel/src/__main__.py at line 5
  [3.29]→[3.0:28](∅→∅)
  − import tensorflowjs as tfjs
• replacement in trainmodel/src/__main__.py at line 13
  [3.8332]→[2.218:367](∅→∅)
  − m.heightmap.fit(x=training_data[0], y=training_data[1], batch_size=100, epochs=5000)

  − tfjs.converters.save_keras_model(m.heightmap, 'tfjs_model')
  [3.8332]

  [3.128]
  + m.heightmap().fit(x=training_data[0], y=training_data[1], batch_size=100, epochs=5000)

  + m.save('models')
• replacement in trainmodel/src/__main__.py at line 18
  [3.8568]→[2.368:429](∅→∅)
  − outputs = tf.reshape(m.heightmap(inputs), (1024, 2048, 1))
  [3.8568]

  [3.8619]
  + outputs = tf.reshape(m.heightmap()(inputs), (1024, 2048, 1))
• edit in trainmodel/Makefile at line 10
  [3.165]→[3.84:143](∅→∅)
  − env PATH=$(PWD)/venv/bin:$(PATH) pip install tensorflowjs