CRYBLOCK 2018 Paper #27 Reviews and Comments =========================================================================== Paper #27 Federated Byzantine Agreement to Ensure Trustworthiness of Digital Manufacturing Platforms Review #27A =========================================================================== Overall merit ------------- 1. Reject Reviewer expertise ------------------ 3. Knowledgeable Paper summary ------------- This paper applies stellar consensus protocol in the NIMBLE collaborative manufacturing platform to ensure the NIMBLE federated instances to achieve the consensus and trust under the federated Byzantine agreement model. Comments for author ------------------- The reason I gave your paper the negative vote is that your paper neither targeted a fundamentally new problem nor proposed exciting techniques. Your motivating scenario is reasonable but it is a straightforward application for distributed ledger technology and I know many blockchain companies are running similar things (but not among different countries). If you could propose a better consensus protocol for FBA model and apply it in your scenario, I think the paper should get accepted; however, what you did was just to simply tweak the stellar consensus protocol and apply it in your scenario, which is not enough even for a workshop paper. More details: - The paper is poorly organized. At least you should highlight your contributions in the first section; otherwise, it is not clear what key research-level contribution you made. In addition, it is difficult to understand the key difference between your work and other existing efforts that apply the state-of-the-art consensus protocols into their blockchain systems. - The motivating scenario is not clear. You should use a small but illustrative example (e.g., the supply chain) to clearly describe the workflow and communication among different instances, and then highlight the key problem the current platform met in reality. - I think section 3 is too long and not quite necessary. A better way for you to compare the state-of-the-art consensus protocols like PoW and PoS is to show the difference, Pros. and Cons. in a table, rather than putting so many words to discuss the advantages/disadvantages. This issue occurred in the following sections again. For example in section 4, you spent a lot of space on talking about FBA background, wasting a lot of space for *your* own technical contributions. I am not saying you should not talk about the background -- you, of course, should and this is a must; however, you spent too much space on that and I cannot see the technical contributions you made. - FBA language -> FBA model/context Review #27B =========================================================================== Overall merit ------------- 2. Weak reject Reviewer expertise ------------------ 2. Some familiarity Paper summary ------------- The authors inspect FBA algorithm (Federated Byzantine Agreement) to validate that the algorithm is doable for the real business model, NIMBLE to ensure trust and reputation between the distributed platform instances. Also, the authors did an inspection of SCP (stellar Consensus Protocol) in order to identify what components need to be added to realize SCP for the federated platforms. Comments for author ------------------- I enjoyed reading this paper. The paper present a great related work by clearly explaining downsides and upsides of the existing algorithms and mechanisms. It is well organized. It will help readers to easily catch the main challenge of the paper and the main reasons why FBA and SCP are feasible and applicable for the federated platforms. The paper details actual examples how FBC and SCP can be applied to a real business model. This is very useful for developers who are looking for techniques/algorithms to ensure trustworthiness between distributed platforms. Although I buy the motivation and the practical value of the paper, the work feels more like engineering (fairly comprehensive though) and it is not clear to me what the research elements are. Not sure there is much novelty in the techniques involved. It seems like a rather straightforward engineering work, without any novel idea in depth. The work is also specific to NIMBLE. It seems more developer efforts/coordination is still needed to apply the finding to other platforms. Review #27C =========================================================================== Overall merit ------------- 3. Weak accept Reviewer expertise ------------------ 2. Some familiarity Paper summary ------------- This paper explores the use of the Stellar Consensus Protocol and the Federated Byzantine Agreement to support the NIMBLE platform and its goal of providing a federated digital cloud manufacturing platform. The paper provides an overview of the challenges of the NIMBLE platform faces as well as of the proposed protocols to address them. The paper also provides a detailed and technical overview of deploying NIMBLE using SCP/FB. Comments for author ------------------- The paper is well-written and addresses an interesting problem. I found the introduction and background sections to be very clear and detailed but a bit lengthy and too simplistic for a reader who is familiar with distributed systems (but the more general audience will definitely appreciate it). While the paper does not contain a new technical contribution, it does a great job of applying well-chosen existing building blocks to NIMBLE. There is a definite overlap between the needs of NIMBLE and what SCP/FBA offer. However, again, there is no new technical contributions here but this submission finally goes beyond "just apply blockchain technology to X and consider it a solution" without actually providing a detailed description of how it would work or could actually be done in practice. Section 4 carefully explores the NIMBE process and aligns it closely with the steps of SCP/FBA. It is done at a level of detail that allows to actually envision the resulting system and reason about its correctness and performance. I wish, however, there was a section discussing the performance of the resulting system (itself and from the participating users perspective). Section 3.2 Community consensus mechanism is confusing. Is its purpose to be a related work section or building blocks section for the current proposal? For example, Tendermint, HoneyBadger are never mentioned anywhere else and are not compared (just described) to SCP/FBA. The figures should be redone to be easier to read (font size, colors and resolution). They are hard to look at in a PDF and almost impossible to read in a printed version. Please keep your terminology more consistent with the literature cited. For example, Well-behaved nodes -> honest nodes Ill-behaved nodes -> dishonest nodes (after defining what honest and dishonest nodes are allowed to do)