Credits

We kindly thank (in random order)

• Artem Babenko and Vladimir Aliev (yandex research) for helpful discussions and editorial review of the paper,
• Jacob R. Steeves (bittensor) for discussions on RPC frameworks and NAT traversal and peer-to-peer technologies.
• Dmitry Afanasiev (yandex) for his guidance on networking and communication technologies,
• Lidi Zheng (google) and grpc-aio contributors for their awesome framework and this pr
• Brian Muller (parallel markets) for his implementations of kademlia and rpcudp
• Alexander Sherbakov (itsoft) for helpful discussions on PC and server component architecture,
• Our early adopters, contributors, and reviewers

Related projects

We also want to reference several projects that have similar ideas in mind:

• BitTensor - a decentralized deep learning ecosystem with with incentive mechanism. Like hivemind, but peers are getting rewarded for their contribution to other peers. (note: as of 26.08.2020 the project is in the early stages development).
• GShard - a paper by Dmitry Lepikhin et al that demonstrate the effectiveness of huge Mixture-of-Experts models on conventional hpc hardware. Those guys train models 4 times the size of GPT-3 on thousands of TPUv3.
• Also doing research in decentralized deep learning? Let us know!