DtCraft FAQ

This page summarizes the list of frequently asked questions.

What is DtCraft in a nutshell?

DtCraft is a distributed execution engine that comes with a unique stream graph programming model to streamline cluster programming. Our stream graph programming API largely reduces the learning curve of building distributed and parallel applications.

How can I apply DtCraft to my jobs?

DtCraft can help scale up and scale out your workload in both effective and efficient manners. Our programming model is very neat and expressive. Most users can master our API within one to two weeks to prarllelize their applications or create new types of workloads.

How does DtCraft relate to existing tools?

DtCraft aims to strike a balance among programmability, performance, and productivity in cluster computing. We offer users transparency without taking away the full control over system details to achieve the best performance. Modern big data tools are good at abstraction but performance, whereas traditional HPC languages are exactly the opposite. Therefore, we are developing DtCraft to bridge the gap.

Who are the targeted users?

We welcome any individuals and organizations who are interested in building products or extending present assets at cluster scale. Whether you are application developers or domain-specific engineers, we provide a new framework for implementing parallel and distributed applications.

What cluster environment do I need?

Currently, DtCraft works on Linux machines. Each machine node must be of 64-bit kernel 3.8 (or higher) and POSIX-compliant, and support C++17 build. Although you can do all file settings by hand in a cluster, we recommend installing a shared storage to access files over a computer network from a local console.

How large a cluster can DtCraft scale to?

We have successfully applied DtCraft to solve a large-scale semiconductor design optimization problem on an Amazon EC2 cluster of 40 machines, each of 4 CPUs and 8 GB RAM. Data are shared via the NFS protocol.

Where can I get more help?

Please post any technical issues on our github issue tracker. For more information, please refer to contact us.

How to cite DtCraft in my work?

Please cite our journal "DtCraft: A High-performance Distributed Execution Engine at Scale", IEEE Transaction on Computer-aided Design of Integrated Circuits and Systems (TCAD), 2018.