Tech + Community Update — June 2024
Here’s an update on our progress in the past month.
Overview
Here’s an update on the three major areas we’ve been working on for Lambda since our last update in May. This work has primarily focused on backend and infrastructure, involving complex technological tasks.
One limiting factor is the development team, which is currently on standby due to ongoing discussions about prioritization, features, and design for the frontend data side of Lambda. This hasn’t halted development but has slowed progress since only one person is currently handling these tasks.
Tech Update
Data Systems
We have had the latest iteration of our data system in place for most of this year. There have been a few instances where market data collection stopped, affecting the accuracy of signals or halting signal generation altogether. This has prompted us to reconsider the systems that feed trading and alerts, as we need far more reliable data systems.
As a solution, we developed the third generation of the data collection system. This robust and tested system will enable the addition of other exchanges in a reliable way.
Before
The previous architecture relied on a parity system and a swarm of workers to retrieve 10s aggregate candle data and 1-minute reported data from exchanges. We used reverse residential proxies to counter exchanges blocking us. The signals system, which primarily relies on the 1-minute reported data, was about ~92% reliable. 8% of the time, the data was delayed by a minute, and there were occasional data gaps lasting several hours.
This infrastructure isn’t entirely obsolete. It is now used to retrieve other metrics such as OI and funding, most of which can’t be retrieved via web sockets or are tedious to retrieve via web sockets. This infrastructure will continue to serve data accuracy and parity checking purposes.
After
The new data collection system has been running smoothly, requiring minimal intervention on the parity side. The constant listeners and data aggregation from web sockets have been faultless. We are currently implementing the final changes for this system, which will bring us to a level of reliability enabling trading and alerts for price and signal-related events.
Major tasks nearing completion include:
- A central register for listeners to obtain a list of assets to monitor via web socket connections to exchanges.
- A suite of workers that can spread over multiple machines to handle web socket connections, retrieving the latest trade data (buy/sell volume, quantity trade sizes, liquidations, etc.) as well as candle data.
- Intermittent automated parity checkers that alert Discord/Telegram in case of a data gap event and retrieve missing data, recalculating signals as needed.
This system is already integrated into the frontend. All real-time data entering Lambda is enabled via this system, and backend data fetching will now adopt this mechanism for trading and alerts.
Trade Engine
We have built and tested a trade engine with four exchanges (Binance, ByBit, KuCoin, and WOO). This engine can function on both the frontend and backend, allowing us to provide a mechanism where exchange API keys will never need to be stored on our secure infrastructure.
We are also actively working on exchange partnerships with integrated exchanges… this will help reduce fees and increase support for Lambda users.
Server-side exchange key storage will be required for automated and signal-based trading and triggers, as well as our trading interface in Telegram, which links directly with the Lambda system in the web application.
We have battle-tested a secure key management infrastructure in a test production environment for over a year. This system, licensed indefinitely to Basis, has performed flawlessly with accurate and safe key retrieval in isolated execution nodes.
The remaining tasks for the trade engine include:
- UI/UX development for trade integration.
- Finalization and thorough testing of Telegram integration.
We have development resources ready and have negotiated a contract to start as soon as the remaining UI/UX priorities are finalized.
Alerts Engine
We have fully scoped and designed a customizable alerts engine since the beginning of the year. All features are specified, and the implementation design is complete. We are waiting to commence the final stages of development once:
- The final data infrastructure is in place (in progress).
- The pending UI/UX priorities are finalized (currently in the final stages of discussion).
This part will mostly be developed by the on-hold development team.
Moving Forward
Our ongoing work on the data systems, trade engine, and alerts engine will significantly enhance the overall performance and reliability of the Lambda platform. The new data collection system will provide more accurate and timely data, ensuring better signal generation and trading accuracy. The trade engine’s development will offer more seamless and secure trading experiences across multiple exchanges. The customizable alerts engine will give users more control over their trading activities, ensuring they can react swiftly to market changes.
These improvements aim to provide a more robust, reliable, and user-friendly experience for all our users, enhancing the value and effectiveness of the Lambda platform.
In the coming weeks and months, we will be focusing on the following areas:
- Finalizing the new data collection system to ensure full reliability for trading and alerts.
- Completing the UI/UX development for trade integration and thoroughly testing the Telegram integration for the trade engine.
- Starting the final development stages of the alerts engine once the data infrastructure and UI/UX priorities are finalized.
We greatly appreciate any constructive questions and will be happy to answer them in text or at the town hall. A channel for questions ahead of next week’s Townhall (June 26th) has been opened in the Lambda Discord Server.
