Trust tensorflow's thread safety (it wasn't the cause of an earlier bug)
[?]
Jul 6, 2021, 10:48 AM
CWOSQTC4HIBTZMMASA34RVBLPT2FRCVGEOIYUZXISNKMDNILV3WQCDependencies
- [2]
TCMUFA6ETry some different hyperparameters - [3]
4IXWCEHMGamma correction - [4]
SJHJS463Model trainer: main function to open sdl window and launch second thread for actual training - [5]
E742MTJAFix segfault (don't pass pointers between functions in different SDL versions), minor refactor - [6]
2ABZP2KNModel trainer: save map.png after training - [7]
ALMG52BOmodel trainer: speed up by waiting 15 seconds between monitor renders - [8]
EB3DTD43Model trainer: use ground altitude instead of crust mass distribution - [9]
ZRPV3GAJModel trainer: Add function to make tensor of cartesian coordinates for drawing the map (equirectangular projection) - [10]
7ML3OFE7Model trainer: initial train and visualize thread pair (total crust mass; todo: altitude instead) - [11]
FL3C6ERZAssume python is being run with -m switch - [12]
ROQCAPZJBegin function for showing the map (for now just opens SDL window) - [*]
ZM2EMAZOStart doing python multi-module stuff properly
Change contents
- edit in trainmodel/src/model.py at line 36[4.2054]→[4.2054:2238](∅→∅),[4.2238]→[2.139:191](∅→∅),[2.191]→[4.2284:2530](∅→∅),[4.2284]→[4.2284:2530](∅→∅)
class Monitor(keras.callbacks.Callback):def __init__(self, target):self.lock = threading.Lock()self.target = targetself.stable = clone_my_model(target)def on_train_batch_end(self, batch, logs=None):self.lock.acquire()self.stable.set_weights(self.target.get_weights())self.lock.release()def snapshot(self):self.lock.acquire()result = self.stableself.lock.release()return result - replacement in trainmodel/src/drawmap.py at line 12
def run(monitor):def run(m): - replacement in trainmodel/src/drawmap.py at line 22
outputs = monitor.snapshot()(inputs)outputs = m(inputs) - replacement in trainmodel/src/__main__.py at line 9
def train(monitor, filename):def train(m, filename): - replacement in trainmodel/src/__main__.py at line 12
monitor.target.fit(x=training_data[0], y=training_data[1], batch_size=100, epochs=5000, callbacks=monitor)m.fit(x=training_data[0], y=training_data[1], batch_size=100, epochs=5000) - replacement in trainmodel/src/__main__.py at line 15
outputs = tf.reshape(monitor.target(inputs), (1024,2048, 1))outputs = tf.reshape(m(inputs), (1024,2048, 1)) - replacement in trainmodel/src/__main__.py at line 28
monitor = model.Monitor(m)bg_thread = threading.Thread(target=train, args=(monitor, sys.argv[1]), kwargs={})bg_thread = threading.Thread(target=train, args=(m, sys.argv[1]), kwargs={}) - replacement in trainmodel/src/__main__.py at line 30
drawmap.run(monitor)drawmap.run(m)