Architecture documentation for Configurator and API (#13935)
* Architecture documentation for the configurator and api * Apply suggestions from code review Co-authored-by: James Young <[email protected]> Co-authored-by: James Young <[email protected]>
This commit is contained in:
1 parent
aeff347a07
commit
9a0118c603
4 files changed
+66
No files matched your search
@@ -19,6 +19,7 @@
|
||||
* [Overview](newbs_building_firmware_configurator.md)
|
||||
* [Step by Step](configurator_step_by_step.md)
|
||||
* [Troubleshooting](configurator_troubleshooting.md)
|
||||
* [Architecture](configurator_architecture.md)
|
||||
* QMK API
|
||||
* [Overview](api_overview.md)
|
||||
* [API Documentation](api_docs.md)
|
||||
|
||||
@@ -0,0 +1,61 @@
|
||||
# QMK Configurator Architecture
|
||||
|
||||
This page describes the web architecture behind QMK Configurator at a high level. If you are interested in the architecture of the QMK Configurator code itself you should start at the [qmk_configurator](https://github.com/qmk/qmk_configurator) repository.
|
||||
|
||||
# Overview
|
||||
|
||||

|
||||
|
||||
# Detailed Description
|
||||
|
||||
QMK Configurator is a [Single Page Application](https://en.wikipedia.org/wiki/Single-page_application) that allows users to create custom keymaps for their QMK-compatible keyboard. They can export JSON representation of their keymaps and compile firmware binaries that can be flashed to their keyboard using a tool like [QMK Toolbox](https://github.com/qmk/qmk_toolbox).
|
||||
|
||||
Configurator gets metadata about keyboards from the Keyboard Metadata store and submits compile requests to the QMK API. The results of those compile requests will be made available on [Digital Ocean Spaces](https://www.digitalocean.com/products/spaces/), an S3-compatible data store.
|
||||
|
||||
## Configurator Frontend
|
||||
|
||||
Address: <https://config.qmk.fm>
|
||||
|
||||
The [Configurator Frontend](https://config.qmk.fm) is compiled into a set of static files that are served by Github Pages. This action happens every time a commit is pushed to the [qmk_configurator `master`](https://github.com/qmk/qmk_configurator) branch. You can view the status of these jobs on the [qmk_configurator actions tab](https://github.com/qmk/qmk_configurator/actions/workflows/build.yml).
|
||||
|
||||
## Keyboard Metadata
|
||||
|
||||
Address: <https://keyboards.qmk.fm>
|
||||
|
||||
The Keyboard Metadata is generated every time a keyboard in [qmk_firmware](https://github.com/qmk/qmk_firmware) changes. The resulting JSON files are uploaded to Spaces and used by Configurator to generate UI for each keyboard. You can view the status of this job on the [qmk_firmware actions tab](https://github.com/qmk/qmk_firmware/actions/workflows/api.yml). If you are a QMK Collaborator you can manually run this job using the `workflow_dispatch` event trigger.
|
||||
|
||||
## QMK API
|
||||
|
||||
Address: <http://api.qmk.fm>
|
||||
|
||||
The QMK API accepts `keymap.json` files for compilation. These are the same files you can use directly with `qmk compile` and `qmk flash`. When a `keymap.json` is submitted the browser will poll the status of the job periodically (every 2 seconds or longer, preferably) until the job has completed. The final status JSON will contain pointers to source and binary downloads for the keymap.
|
||||
|
||||
QMK API always presents the source and binary downloads side-by-side to comply with the GPL.
|
||||
|
||||
There are 3 non-error status responses from the API-
|
||||
|
||||
1. Compile Job Queued
|
||||
2. Compile Job Running
|
||||
3. Compile Job Finished
|
||||
|
||||
### Compile Job Queued
|
||||
|
||||
This status indicates that the job has not yet been picked up by a [QMK Compiler](#qmk-compiler) node. Configurator shows this status as "Waiting for an oven".
|
||||
|
||||
### Compile Job Running
|
||||
|
||||
This status indicates that the job has started compiling. Configurator shows this status as "Baking".
|
||||
|
||||
### Compile Job Finished
|
||||
|
||||
This status indicates that the job has completed. There will be keys in the status JSON for source and binary downloads.
|
||||
|
||||
## Redis/RQ
|
||||
|
||||
QMK API uses RQ to distribute jobs to the available [QMK Compiler](#qmk-compiler) nodes. When a `keymap.json` is received it's put into the RQ queue, where a `qmk_compiler` node will pick it up from.
|
||||
|
||||
## QMK Compiler
|
||||
|
||||
[QMK Compiler](https://github.com/qmk/qmk_compiler) is what actually performs the compilation of the `keymap.json`. It does so by checking out the requested `qmk_firmware` branch, running `qmk compile keymap.json`, and then uploading the resulting source and binary to Digital Ocean Spaces.
|
||||
|
||||
When users download their source/binary, API will redirect them to the authenticated Spaces download URL.
|
||||
@@ -0,0 +1 @@
|
||||
<mxfile host="Electron" modified="2021-08-09T19:46:29.036Z" agent="5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) draw.io/14.6.13 Chrome/89.0.4389.128 Electron/12.0.7 Safari/537.36" etag="PQ2r34UrZa0TfW4Fw0EV" version="14.6.13" type="device"><diagram id="NEtccoSKIy4HskWlhJpu" name="Page-1">5VvbcqM4EP2a1O4+hOLqy2Ni5zKX1CTxzszOU0oG2dZEIBZEYu/XbwuEDQg7csZ2vFlXjQca0RLdR+eohXPiDML5VYLi2Q0LMD2xzWB+4gxPbNvqezb8JywLaTFtr7BMExJI28owIv/gsqG0ZiTAaa0hZ4xyEteNPosi7POaDSUJe643mzBa7zVGU6wYRj6iqvU7CfissPY8c2W/xmQ6K3u2THklRGVjaUhnKGDPFZNzceIMEsZ4cRTOB5iK6JVxKe67XHN1ObAER1znhgfT76Zzxxp3Z49j5+7zjw/z+NSS+XlCNJNPLEfLF2UIcAARkacs4TM2ZRGiFyvrecKyKMCiHxPOVm0+MxaD0QLjT8z5QqYXZZyBacZDKq/iKDgTyYLTiEW4sFwSSqXLAKWz3L9orD64jEXKssTHG57WlQBCyRTzTVGRDsWTV3qQcb3CLMQ8WUCDBFPEyVMdK0hCbrpst8oKHMjEbJEk9+UckTDHcTWm0jQk4RR6pWQM3z4l8QNKuDhkYZxxnKRw/J0ljymHJ2HRg2X35vDPiKPp2kA/4YTj+cbIyKunjmv0zMpHeljSQXH6XJla0jSrzKrStvPIei9Hdgm9HNoZpyTCgyXZmPWYI0qmERz7EDGcgIGiMaa3LCUittULIoQEaOZzo8GYcc7CSoMz6ZKLqXQOJBKLgYXzqSBc4xmPKcy11JgSPsvG0GLCIn6JQkJFiK8xfcLCjbwgJyDM+uJ8wChL8ud0ivSILnjCHnHblQlMyIp9kn/ADkMJCK55G14OL4YXmybrFhhyOzXQtKEGdMVwVeBUzDvHTjmKjdS54jWfojQlfpP3YCR/CRwZXnn6o3ptOJcgK84W8gwma8JVz7lZ0qZVI9Ff401PkzddTdqsJM5rme6lTZtdZQ+3jMCTLXFjlRkqyaZJI8Vzy7uqytl01Gk4chqOisAojnJYLR/7F5BWTpkV0gZUTDjgbnOUxTEorqCosxO7QyGG5+MEjqbiaDBLoPsTGzo2L0mCJ2yuNhri9FHQi21+iehCAfHzjHA8ilEOkWegnTqGm0QVkiDIVwU5950j/3Garw9Kdsjd7YYZlmurkhlUYrDNFoB19qUnltPCCUW40xhFZcBnnIvl65noTChxNCFT4+/w0ZiEZXOwV+9Qk9bmd0SiKYTeNm+F9AM2RKTX3tno4iN6QiM/IbFYHXz7eqHeOJI8AeM2YbTQDL4fiuFnCeIs0RvmNUt54eWqUK1iwOnaZ28Asr7cfC0894E/q2PYehDs7w2CGqtFdbndqklVRaoI1BpN2tEaXVdryql2JGKzTKiEgvdasXHchqNDi01XAz/vpiK0tOHW0YTbYUpC6/1ULj9TuH0XlWZ93riOuhjotsz/rmf09pWkjpKkuvI/4sWYoSRIG+K/ks9PsgV4ucEcBYgjXV2ekCR8hoiq7a+uoeGZX+RN3PQ1BsdiwXB2+wG+h629rFR7SKDaRPSLj4VIm7n2ptsuG2voPKRIu7Zt9N9apHtbiXRr7VirASVHVgtAc4cFoDZNekclypZnWK5p9Tp9y+v1up1y73nRQMbWBWHdz3LP90ASbav14HuW6L4m9spsHotE95Us3d18Kkm2mS5KSZziik76lGXBy+XNLpTTq8O5Y6rKadkts3Rv27JlZ2uFE8VkrWTeX4z+nGS0DLSeWILDTYp3jyJ/Bqsf2xRrFXP4Bb6+3fy2tegdUuc6pqenct29pVFjkfryDulhtzDLWXskCnbarCtd75WadaoUlpqqBelBi0qzWDRINwzZanTkOObmkTVv8Pq1G+CgGMNuRVSl53csonZXV0R7RyWitrobIasPMMoCRGF4WY807b+PnD+OS3idXh347fuHrdrbt8qWu4+5WpzU1RcE81QocLTwHcMPIiMoUsJEPtI8+obPWqRZRlNK7+kLoqskejut3UWCeg1msh3L6PYrn54qsf2WbO3thYOjrpTeTGK1Oea4ikSQ/EaSX7t1qwiZfeC9W6ft9dO71TSn/D3YS3hz7KPSNEct3+/vlDyVciS2/FLOEnwYSWpskXS7ai3YOSjB2WqwcEDEm+9cjEwRuqaGfGTineJdhrOW3c/2AjApnO6/BNxBkpqVXrfz1pWes91Lx73KkPZ+UTkP31qHyiw26zFt2fEacGiWiPtWHXWNXvwuIIwpETNmDbPhuZhhAcvG+RWrJj3Wm2x99Vt+QXLYrS9XXdAVu4YDCCehIpztbHefRVH+a07dX22sc/efoLy+/daU5+pswh+I8vRXQt3joDyZVbeW015ztaxLgHU3/eZbnT3Tn6uuULYGwoF/z+lo7wYdF16AHes00GuuO/V3Qxui2bP0ULP1bqjTYK5yS2Gvm5vu/6sQ1IVzKa67KwTXEJK3GVu6KLWd14H0ZRjB6eovj4rmqz/gci7+BQ==</diagram></mxfile>
|
||||
@@ -0,0 +1,3 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" version="1.1" width="1261px" height="991px" viewBox="-0.5 -0.5 1261 991" style="background-color: rgb(255, 255, 255);"><defs/><g><path d="M 885.2 40 L 1060 40" fill="none" stroke="#000000" stroke-miterlimit="10" stroke-dasharray="3 3" pointer-events="stroke"/><image x="804.7" y="-0.5" width="80" height="80" xlink:href="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAHcAAAB8CAYAAABT/i9JAAAgAElEQVR4Xu19CbhdZXnuu/Y8n3k+OUPmhAyQQMJMQCZxqFpBvU7VKk6otQIq4BBBLVQF7a2ot/a2trW3j09br+3F1lZAJBBIQkLInJOcedjD2WfPwxrv8/3/+tdee+eckIRDCMcsnnCmvdfee73r/Yb3+77/l3D+OKMr0NbZ8xmfz/95p9PVkEonP5SMTf7zGZ3oVXyS9Cqee8GdurV10RKP3/tIMBS+0e3xegzDAP3TNA3pmcQTU+PD1wPQz5UPfh7cU0Cis6fvM35/6PPBcF0PXTACVNd19lWWy8hnZiBJBnKZ9NJ0On3sFE55Vh5yHtw5LnNbb29/wO3/fjBcd6PX5/MYugGdmGqCmkknIZcKcLlcYIDDQHomeR7cs3LbnuGL9PYvuSMQjNwVqqvvkSBxQA0dBG6pXER2ZpqB6XA5icKMvfSPvk+nzoN7hpf91Xtab29vv8cf/n64rv5Gnz9QxVLd0JFOJlAqFuBymyy1g0qcZdgayJwH99UD6XTP3L985R3BYPiu+vqmHkmqZmmxWEAqGWcG1+kklnIA6T9AgivYjIBHwkx8gjMYBO7MebN8uiDM5+N7e1f0B8OB74XrG28KBAMeXTdNq66zIGl6OoZSPgeX2819qQkcM71OP3zNfWjoXgU5n4YrN4TxwUOWWc6kz4M7n1id8rmWr1x9RzBSf2dDY0uvwyGBg0qAGigW8piOT7FzMZYyU8v/SZIL4dZeRDpWQPLVM4YSzvmZKbgZuAfNxwLZ8+CeMh6v+IG9K1b0hwOR7zU2Nt0UCIYqLDV0aJqORGwC+VwOHo+HvZYAlL66A/Vo6V2Ftv41cDg9KJZkZDIZ6Dx2QoHAzQ9h/LgJLgxk06nzZvkVo/YyJ1i5dv0ddZGGOxubW3slh8QiXcHSfD6H+NQEIJEvdVVFvJLkQF3HEnQsXofG1m4oGt0EGlRVR0kuI5PNmcETZ64nP4QxE1z6QzZzHtxXBdsVK9b1ByKB7zU1td4UDIc9PCel3FSHrumYmhhDPp+Bxy1YWvGnHn8EnUvXo2vxGji9PsZqTdVs4GoolRXkcjnOXBBzJ+HJD2Ps+AGL8edFjHmG9oILN3yqvr7xrpbW9l7J4aiw1CDFKIPo5Djzky6nqyo4oui4qXMpelduQFP7IqgalxFVk6nse12HqumMucVSGYVCocLc1CS8BO4xE1ycV6jmBdoV69b1RwJ1jzS3tt0citR5hMggWDoxNoxsJgOP12ulKcKf+gIRBmjv8rVwewMWeKqucoBVjTGXg0xgc/9cKJVQKpUrPtcEd/TYfitNymXPy49nDPCFGzZ/qr6p8a7Wts5eh5OzVChImVQKk+OjjKXXfOh+/Oaf/hKeUtxMU4C23mVYesElaOvuhaYbjI0MQAaiyVrxO12DRt/rOjSFMziXL0BRVM5cGCikphhzCVyhUOWymfMB1emgu27duv5AXdMjbe0dN4cZSyu+VFNVjA4PIZtJw+PzWcHRRW//LJ79xU8QCRhYdsElWLZ6A9xeP2MisdAeKHHTq3K2EqCqDk3nQRQBr+k6FFVDPl9gj+HRsoFiagqewjDGBji49F/+PLinBu2GzZd9qqGp5c6Oju4+J7HUVolJzyQxOjrMzCEJ99zkckaR8NDRuwyrL7wUnT39jKUc0IqJJbAoyJJNs0vfMxNMTNU0BiYPqkyfq2nI5nLs7OK1BLijA/vMmwrI584zd050N27c2BOINP5FW0fXTXX1DV6RvtBXYung8WPIpNPw2ljK5EDDgD8QxqqLLsXqdRvgDQQ5A22+U2GMNEEUv9cMzk4RGdvYave5ZI6LxRK7gbgbAIrpKXgLwyBwhVnO57LnzXItupsvu/pTTa0td3Z0L+pzOpxmaY3nptOJBEaHB7meSyy1q0eQsKh/OdZtuhy9vUu5L2UAqqZZ5X6zApQAU/yO/m76VxNwzlx+IxBzFV1DuVyGLJv+VoBLPrc4jNGjHFzmh8+Dy6HduPGKnkhj5C/auxfd1FDfyFlq1ksVVcXxo0eQTs3A6/NXqUeMpcEA1l9yJdZvuIQxlgdGnJVkfjViJDO5ZJJV5mPZ36tSHf547mvNnNZkNmO4Qv6W+95CscTlSlN6JL9LZtlXHMbI0ZcsX1/I536/mXv5Vdd9orm19e6unt4+l7OapYlEHEPHjgGSyVJbaY1qq71Ll+OSy67C4iXL2I0gIlpimshPK0ALQCnFqfhP5nuFiRZ+mBhKQJsstm4QAldRbSmQKO9xs+wrDGNk4CXLLP9egrvxiit6muoav9+5qOfmhoZmL+WjPOrVISsKBg4fQkqw1AKUqb3wBf245LKrccmmyxAMmixljDM4I8kEM8bZgiZbIMQZa/paK4etAC9ukEpqxNUp4a/LZQWKRia5Aixn7iT8xRGMHN1rBXPF3yfmXnXd9Z9obWu7e1Hv4j5SiOySYCw6hcFjA9TTUIl4TX8KQ8fiZStw5TXXYfnyFeziMT9oRb3cT3Jwud9UFDNgIp8pclcC3jLJFXYL9Ymdj8w3Oze/STioHGD6yoQLK5/m0iMLqKrA5RWkYiG/sM3yFVdc3xNuCn+/p6fv5sbmFq9oJBMsPXRgP2aSSfj8gSr1iEe8flx+1RZcfsWViETCDEBZsI5FtjxQ4nmq6StZ4MN9JmdfdZQs5EPL32oGT4PMKLk6guapEGezCkVWURbCRRVzDZTSU4y5w0f2WiXCBQ/uO9793rGGxqau1rYOKzedmpjAwNEjoKoLtaqIthSRQixZtgw33HgjVq9aRaS1BUAUDNnVJGKaWakxWSdECZbSMNZSMGQKEJZgMVt0LExvJdAiUO1gC9baAynx3jm4wwxc8TmKxcLCZu77PvyxeG//4uax0WHEo1FMT0/D5w9a2Y+4EJSrXnXNNbj22mvQ3NDIGMejWm5KeZAk2MhlQvY7FslW/KiIaK2giokWFSGCsZydi7OegWfLf4VvFWaeiRfEWkWDLCuWImX2wDGzTO6FwA3YwKXPVVro4L73Qx+N9/QtbiY0H/+vX3Pzazv6+vpww0034sK1a+GiDkK6WEziMwE10w8R1VI6Q6kRD5ooiCKpUKQ6Zgpk/syrOmbka/pny4eKn0VkzEAUNw/dUDxI48qUipKssL/XBlLiZwZuaRhDh1+0UrVUsbC0vJD7lt/zwY/Ee/r6GbhP/ua/q8D90r33oG9RdxXYdLE4YwkwzlpZ5TVY+j2XDSklqWay8LuCpZXASQBuD4wq0qMIwKwcl4HKH0vn1Ml/qypjrV2REoGUJT+mpxA0wRXWKFUqLmxw3/3+D8cXmeA+9cTjVeA+/N3vIBDgooQ4GAs1jQVOQh0SOSsPkipiv/V7M4iqmG0TGDNStgsT9BzdZL+9aCAKBdxPc/B1M/ImRUrTeVRczVyhYYOb5dIwhg+/aBXr06ni0nJ5AU8c3Pa+D8YX9XLmPv3bJyv+1jDw8MPfrQKXfJeIcJlPNVMYlp7Y81NmqvlNYEXLTOgXZtgWITOgeF22okqZ1R5xA9mKCKKNhqVBZI7LMmOyKERw9p4INIkYwfIwhg7tMaNlYMGDe+t7PxDv7ulj4G576in4AzyYIgY8UgOuAJD5VMZgEUjZgDHLcNznmt0RigbF9JeceVxmtAdglRJe5SawgLSlSyyNMvNbRVFs5riatSKQEoEVMVeAK8xyOp1aWi6XF+6s0Dvf/f54V08vA/fZbU/PCS4rxZkRseVXmVBfqavatWAC154GCVZyEcIwdWAOFPPXImUywWMmV3RWWDcK5bRmMUGjVhqhIdea4xOBrgWXAM4sdHDf/q73xrsXcXCfe/YZBq64sx955GHLLFdYWmEcz2lNFpPqRP6WRcgiIOK/48BWQKkKuMwAjD2PtGJR7WHacSXFYmKFGUyRby/LZVYsEMwUpb3aQEqY6LLF3N1WpSqTTi9s5r7ttv8R7+ruYeDueG67BS4B/D0TXJH6UDAlCuaWAFFlmm0yo5WbVoQGqwjAmtkoZeJF+EogZhbgrZaaSquq1QBH1kJRGbhz+dfZgC6lowiVhzF4aLdVFcpkFji4b33nuy1wd+143oyWeZRJ4Pr9/irBgphFqY9dThTlOqvSY9ZVrSBJiBmW6eV1V2Hm7W0yXDfm6RSPlrnIIfwvPZbMMd1oFaGCfz9bIMVbWw2UGbhDDFx6LP1vwYP7lne8K97ZvYgxd/eunQxcYZa/971H4PN5udIkhH+rX6kmmLKxzfKvTPcV5pgrViw3NWVHDj4PzERfVJV2bPZH2W+SsixDUXlOWwHUDnSl+8ICnylUUYTlIQwerJjlbCazsM3ym99+W7yjq5uB++LuXSwVEm2lBK7b4zEBsBUBqoA20x2rTYYLGcJ3iuqQAFAUCuzFBMHSSiHBrOmaUTI7HxUlFBlU0mP9WWaz+VyKFE+NKkATcwnc4wcrZjmbXeDg3vIH77TA3btnN/zEXLOU9/DDj7C5HCFYCKmQpzBmMGX6Tt7+woMpxjQKhsyynuWfRRXIVLaEbmw9h4EpqkmC8Sprx5FZELine truncated
|
||||
|
After Width: | Height: | Size: 39 KiB |
Reference in new issue
Block a user