Commit aa801add authored by homiexie's avatar homiexie
Browse files

Merge branch 'master' of github.com:WeBankPartners/wecube-platform into 1183_enhance_model_graph

Showing with 984 additions and 301 deletions
+984 -301
# WeCube
[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
![](https://img.shields.io/github/v/tag/WeBankPartners/wecube-platform.svg)
![](https://img.shields.io/github/languages/code-size/WeBankPartners/wecube-platform.svg)
![](https://img.shields.io/badge/language-java-orange.svg)
![](https://img.shields.io/badge/language-vue-green.svg)
English / [中文](README_CN.md)
中文 / [English](README_EN.md)
## Introduction
WeCube is an open source, one-stop tool for architecture management and IT operation, which aims to simplify the IT management of distributed architecture. Its features can be easily extended by plugins.
## 引言
WeCube是一套开源的,一站式IT架构管理和运维管理工具,主要用于简化分布式架构IT管理,并可以通过插件进行功能扩展。
## Origin
During the implementation of distributed architecture, core banking systems are built in WeBank. Comparing this to traditional monolithic architecture, there are quite a few different pain points, e.g. increasing number of servers, complicated deployment process, difficulties in service tracing and problem diagnosis due to complex systems and long invocation chains.
## 起源
微众银行在分布式架构实践的过程中,发现将银行核心系统构建于分布式架构之上,会遇到一些与传统单体应用不同的痛点(例如,服务器增多,部署难度大;调用链长,全链路跟踪困难; 系统复杂,问题定位时间长等),在逐步解决这些痛点的过程中,总结了一套IT管理的方法论和最佳实践,并研发了与之配套的IT管理工具体系。WeCube就是将该套方法论和最佳实践,从微众内部众多IT管理工具体系中提炼出来,整合成一套开箱即用的IT管理解决方案。
In the process of finding the solutions for these pain points, we have come up with a set of methodology and best practices for IT management in such scenarios and also developed necessary supporting tools. WeCube is the result of extracting the methodology and best practices form WeBank's internal tools, which is then packaged into an integrated out-of-box solution for IT management.
## 设计理念
![WeCube设计理念](wecube-wiki/images/wecube_design.png)
## Design Concept
![WeCube Design Concept](wecube-wiki/images/wecube_design_en.png)
WeCube的设计理念与IT系统生命周期管理基本一致。可以通过“六个维度和一个核心”来阐述。
WeCube's design concept basically matches the lifecycle of IT system management and can be elaborated as "6 aspects with 1 core".
- 1st aspect: "Workbench", it is an ITSM-like workbench that can aggregates all manual tasks so that their contents, importance and priorities are all clear at first glance.
- 第一个维度:Workbench,汇聚任务、统一平台。就是将需要人员执行的任务全部汇聚在一个平台上。这样工作内容清晰可见,轻重缓急一目了然。 类似ITSM。
- 第二个维度:Expectation,规划设计、描绘期望。就是定义模型和规范,形成标准化设计语言。通过规范化设计及图形化展示,清晰、准确描绘出对分布式架构的期望。
- 第三个维度:Execution,自动执行、驱动实现。就是通过各类自动化、标准化任务的执行,将期望设计变成现实存在,消除人员能力参差不齐导致的实现差异。
- 第四个维度:Watching,继续监测、发现差异。就是定义全方位的监测项指标,通过持续收集监测数据,精确反映现实存在的现状,并发现现实与期望的差异项。
- 第五个维度:Wisdom,赋予智慧、制定策略。就是应用机器学习等技术,赋予We3智慧的能力。通过数学建模,制定应差异项的处理策略。 比如资源扩容策略、故障处理策略。
- 第六个维度:Equilibration,不断调整、保持对等。就是通过执行处理策略来不断进行动态调整,最终保持现实与设计的对等,进而使系统稳定运行。
- 一个核心:Workflow Engine,流程引擎、协同驱动。通过工作流引擎赋予We3协同驱动能力,从而高效组织以上六个维度组件的协同运行,减少人工处理环节,提高团队工作效率。
- 2nd aspect: "Expectation", it is to plan, to design, to describe the expectations with models and specifications. By using standard and formal design language together with graphic visualizations, we try to make expectations on distributed architecture accurately and clearly.
- 3rd aspect: "Execution", it is to execute tasks in automated and standardized way so that expectations become realities without introducing errors or deviations caused by people in this process.
## 技术实现
WeCube分为核心功能模块和插件模块。
- 4th aspect: "Watching", it is to uncover differences between realities and expectations by defining all-round metrics which can reflect the varying runtime situation, then consistently collecting and keeping track of the information.
核心功能模块使用Java/MySQL/VUE开发,主要负责工作流程,可视化和核心数据存储。
- 5th aspect: "Wisdom", it is to add intelligence into WeCube by means of data modeling and machine learning, so that policies can be developed to deal with errors and deviations, such as policies for trouble shooting, cause analysis, resource scaling.
插件模块用于功能扩展,分为资源管理、数据整合、功能增强三类插件。对于插件,WeCube定义了一套接口规范,插件开发者可以在遵循规范的前提下,自行选择开发语言。目前已有的插件使用的开发语言包括GO语言和Java。
- 6th aspect: "Equilibration", it is to ensure that realities are conforming to design expectations by applying policies and making consistent adjustments, so that running systems are stable in a dynamic way.
- the core: "Workflow Engine", it is to coordinate tasks and improve collaborations in those 6 aspects, so that manual intervention is reduced and team efficency is increased.
## 系统架构
### WeCube 1.x版本
WeCube1.0版本已经完成开发,整体架构如下图:
## Implementations
WeCube is composed of the core framework and plugins for feature exetnsions.
![WeCube1.0整体逻辑架构](wecube-wiki/images/wecube_arch_1.png)
The core framework is developed with Java/MySQL/Vue.js and reponsible for workflow management, data visualization and core data persistence.
- WeCube1.0版本以WeCMDB为数据核心,配置数据都存储在WeCMDB中;
- WeCube1.0版本需要CAS服务提供鉴权能力, 为保持权限数据的一致性, 建议与WeCMDB共用一个CAS服务。在WeCube1.1版本,会增加支持本地用户验证模式;
- WeCube1.0版本需要一个对象存储服务(S3协议),用于存储物料包、执行脚本等。
- WeCube1.0版本支持了两个插件:腾讯云插件、SaltStack插件。
Plugins are introduced for feature extensions, they can be grouped into 3 categories: resource management, data integration and practice enhancement. By conforming to the interface specification defined by WeCube, plugin developers can choose their preferred programing languages. Existing plugins commonly use Go and Java.
WeCube1.0版本已实现功能见下图:
![WeCube1.0功能架构](wecube-wiki/images/wecube_arch_2.png)
## System Architecture
### WeCube 2.0版本
WeCube 2.0版本正在设计进行中, 敬请期待。
### WeCube Version 1.x
WeCube version 1.0 is released, its architecture is elaborated as follows:
![WeCube1.0 Architecture](wecube-wiki/images/wecube_arch_1_en.png)
## 主要功能简介
WeCube的功能菜单设计与设计理念保持一致。分别是任务、设计、执行、监测、智慧、调整、协同、系统。
- WeCube version 1.0 uses WeCMDB for data persistence, configuration data are all managed by WeCMDB.
- WeCube version 1.0 requires a CAS server for authentication, in order to keep data permissions consistent, we suggest to share the same CAS server used by WeCMDB. In WeCube version 1.1, a local user based authentication mode is supported.
- WeCube version 1.0 requires a S3-compatible object storge service for artifacts and scripts.
- WeCube version 1.0 includes 2 plugins: Tencent Cloud plugin for resource mangement and SaltStack plugin for automated deployment.
- 系统
Implemented features in WeCube Version 1.0:
- 系统参数:管理WeCube Platform运行所需的系统参数。
- 资源管理:管理WeCube提供的资源如容器母机及资源上运行的实例。
- 权限管理:管理WeCube Platform的用户,角色和菜单,可以对权限进行菜单级别的管控。
![WeCube1.0 Logical Architecture](wecube-wiki/images/wecube_arch_2_en.png)
- 协同
- 插件注册:选择插件包上传,插件包需声明本插件的依赖、所需菜单、数据模型、系统参数、权限设定及运行资源,注册后通过容器运行,支持多实例,可以查看插件运行的日志。选择插件服务,通过插件运行的参数关联CMDB数据模型的属性值,形成注册。
### WeCube Version 2.0
WeCube Version 2.0 is under active development and will come soon.
- 任务编排:比如设计一个VPC创建的编排。包括创建VPC、创建子网、创建VM。流程的每个执行节点需要关联插件。
- 设计(WeCMDB插件)
## Main Features
The main menu of WeCube matches its design concept, including Workbench, Expectation, Execution, Watching, Wisdom, Equilibration, Workflow and System.
- 规划设计:用于设计机房结构。
- System
- CMDB model design: support customization and graphic visualization.
- authorization: row-based data permission control for CRUD operations. For example, DBA can only operate on DB hosts instead of application hosts in case that all hosts share one single CI type.
- system parameters
- 资源规划:用于实例化一个机房,特别是两地三中心结构。
- Workflow
- plugin registration: to upload plugin packages and create their runtime environments in containers.
- task orchestration: to design workflows to finish automated or semi-automated tasks, such as task orchestration to create VPC, subnet and virtual machines on public cloud. Each node in the workflow is associated with a plugin for execution.
- service channel
- 应用架构设计:用于设计一个应用的逻辑架构。
- Expectation
- data center design: to create a plan and to design the sturcture of data centers
- data center resource plan: to instanciate a data center design and plan actual resources for a specific data center
- application architecture design: to design logical architecture of applications
- application deployment design: to design the deployment architectural view and trigger deployment for applications
- CI data query and management: to query and manage data for a single CI type through the graph of CI model
- CI integrated query: to configure queries involving multiple CI types, for example to find out all hosts used by a specific application
- enumeration query and danagement
- 应用部署设计:用于实际部署一个应用。支持灰度发布。
- Execution
- artifact management: to manage deployment packages for applications. Customization of configuration file and value substitution rules is supported to eliminate manual intervention.
- application deployment: to view deployment history, to skip or retry deployment tasks.
- batch execution: to execute simple operation tasks in a selected scope of targets.
- high risk commands: to define high risk commands, so they will be blocked or extra approval process will be raised.
- task orchestration execution: to execute complicated operations with predefined workflow and plugins.
- CI数据管理/查询:通过模型图形进入单个数据管理以及查询。
- Tasks (2nd half of 2019)
- CI综合查询管理/数据综合查询:用于配制多CI属性报表。比如一个应用使用到了哪些主机。
- Watching (2nd half of 2019)
- 枚举数据管理/查询:通过对公共枚举和私有枚举进行管理以及查询。
- Equilibration (2nd half of 2019)
- Wisdom (2nd half of 2019)
- 执行
- 编排任务执行:在选择编排后可对选择目标对象执行编排,支持灰度操作。比如重启某个资源集的5台主机。
- 物料管理(Artifacts插件):管理应用程序的包。可以定义各种文件,可以配置环境差异导致的变量替换规则,不需要人工处理。
## Quick Start
WeCube is using containerized deployment.
- 批量执行:通过配置综合查询并选择目标。在通过特定插件来执行任务。比如某个应用的所有主机,执行一个用户权限变更。
PLease refer to [WeCube Compilation Guide](wecube-wiki/docs/install/wecube_compile_guide_en.md) on how to compile WeCube.
- 任务(Service-Management插件)
PLease refer to [WeCube Installation Guide](wecube-wiki/docs/install/wecube_install_guide_en.md) on how to install WeCube.
- 模板管理:服务目录管理, 服务请求模板管理;
- 服务管理:服务请求管理,任务管理;
- 监测(Open-Monitor插件)
## User Manual
Please refer to the [WeCube User Guide](wecube-wiki/docs/manual/wecube_user_guide.md) for guides on WeCube usage and operation.
- Agent管理: 注册、启动、停止;
- 数据管理: 提供数据采集配置, 数据查询等功能;
- 告警管理: 提供阈值配置、日志监控、告警触发等功能;
- 视图管理: 提供图形配置和自定义视图功能;
- 调整(规划中)
## Developer Documentation
WeCube is developed with Java and Vue.js, it uses MySQL for data persistence and relies on Tomcat as the web application container.
- 智慧(规划中)
Please refer to the [WeCube Developer Guide](wecube-wiki/docs/developer/wecube_developer_guide_en.md) on how to set up the development environment.
## 快速入门
WeCube采用容器化部署。
如何编译WeCube,请查看以下文档
[WeCube编译文档](wecube-wiki/docs/install/wecube_compile_guide.md)
如何安装WeCube, 请查看以下文档
[WeCube部署文档](wecube-wiki/docs/install/wecube_install_guide.md)
## 用户手册
更多关于WeCube的使用和操作说明, 请查看以下文档
[WeCube用户手册](wecube-wiki/docs/manual/wecube_user_guide.md)
## 开发者文档
WeCube使用Java和VUE进行开发,数据存储于MySQL,并依赖Tomcat Web容器运行。
请参考以下文档进行开发环境配置[WeCube开发环境配置](wecube-wiki/docs/developer/wecube_developer_guide.md)
## License
WeCube is licensed uner the Apache License Version 2.0, please refer to [LICENSE](LICENSE) for details.
WeCube是基于 Apache License 2.0 协议, 详情请参考 [LICENSE](LICENSE)
## Community
- For quick response, please [raise an issue](https://github.com/WeBankPartners/wecube-platform/issues/new/choose) to us, or you can also scan the following QR code to join our community, we will provide feedback as quickly as we can.
## 社区
- 如果您想得到最快的响应,请给我们提[Issue](https://github.com/WeBankPartners/wecube-platform/issues/new/choose)或扫描下面的二维码,我们会第一时间反馈。
<div align="left">
<img src="wecube-wiki/images/wecube_qr_code.png" height="200" width="200">
</div>
<div align="left">
<img src="wecube-wiki/images/wecube_qr_code.png" height="200" width="200">
</div>
- Contact us: fintech@webank.com
- 联系我们:fintech@webank.com
# WeCube
[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
![](https://img.shields.io/badge/language-java-orange.svg)
![](https://img.shields.io/badge/language-vue-green.svg)
中文 / [English](README.md)
## 引言
WeCube是一套开源的,一站式IT架构管理和运维管理工具,主要用于简化分布式架构IT管理,并可以通过插件进行功能扩展。
## 起源
微众银行在分布式架构实践的过程中,发现将银行核心系统构建于分布式架构之上,会遇到一些与传统单体应用不同的痛点(例如,服务器增多,部署难度大;调用链长,全链路跟踪困难; 系统复杂,问题定位时间长等),在逐步解决这些痛点的过程中,总结了一套IT管理的方法论和最佳实践,并研发了与之配套的IT管理工具体系。WeCube就是将该套方法论和最佳实践,从微众内部众多IT管理工具体系中提炼出来,整合成一套开箱即用的IT管理解决方案。
## 设计理念
![WeCube设计理念](wecube-wiki/images/wecube_design.png)
WeCube的设计理念与IT系统生命周期管理基本一致。可以通过“六个维度和一个核心”来阐述。
- 第一个维度:Workbench,汇聚任务、统一平台。就是将需要人员执行的任务全部汇聚在一个平台上。这样工作内容清晰可见,轻重缓急一目了然。 类似ITSM。
- 第二个维度:Expectation,规划设计、描绘期望。就是定义模型和规范,形成标准化设计语言。通过规范化设计及图形化展示,清晰、准确描绘出对分布式架构的期望。
- 第三个维度:Execution,自动执行、驱动实现。就是通过各类自动化、标准化任务的执行,将期望设计变成现实存在,消除人员能力参差不齐导致的实现差异。
- 第四个维度:Watching,继续监测、发现差异。就是定义全方位的监测项指标,通过持续收集监测数据,精确反映现实存在的现状,并发现现实与期望的差异项。
- 第五个维度:Wisdom,赋予智慧、制定策略。就是应用机器学习等技术,赋予We3智慧的能力。通过数学建模,制定应差异项的处理策略。 比如资源扩容策略、故障处理策略。
- 第六个维度:Equilibration,不断调整、保持对等。就是通过执行处理策略来不断进行动态调整,最终保持现实与设计的对等,进而使系统稳定运行。
- 一个核心:Workflow Engine,流程引擎、协同驱动。通过工作流引擎赋予We3协同驱动能力,从而高效组织以上六个维度组件的协同运行,减少人工处理环节,提高团队工作效率。
## 技术实现
WeCube分为核心功能模块和插件模块。
核心功能模块使用Java/MySQL/VUE开发,主要负责工作流程,可视化和核心数据存储。
插件模块用于功能扩展,分为资源管理、数据整合、功能增强三类插件。对于插件,WeCube定义了一套接口规范,插件开发者可以在遵循规范的前提下,自行选择开发语言。目前已有的插件使用的开发语言包括GO语言和Java。
## 系统架构
### WeCube 1.x版本
WeCube1.0版本已经完成开发,整体架构如下图:
![WeCube1.0整体逻辑架构](wecube-wiki/images/wecube_arch_1.png)
- WeCube1.0版本以WeCMDB为数据核心,配置数据都存储在WeCMDB中;
- WeCube1.0版本需要CAS服务提供鉴权能力, 为保持权限数据的一致性, 建议与WeCMDB共用一个CAS服务。在WeCube1.1版本,会增加支持本地用户验证模式;
- WeCube1.0版本需要一个对象存储服务(S3协议),用于存储物料包、执行脚本等。
- WeCube1.0版本支持了两个插件:腾讯云插件、SaltStack插件。
WeCube1.0版本已实现功能见下图:
![WeCube1.0功能架构](wecube-wiki/images/wecube_arch_2.png)
### WeCube 2.0版本
WeCube 2.0版本正在设计进行中, 敬请期待。
## 主要功能简介
WeCube的功能菜单设计与设计理念保持一致。分别是任务、设计、执行、监测、智慧、调整、协同、系统。
- 系统
CMDB模型设计, 是系统管理的核心功能,目前WeCube的CMDB模型设计支持完全自定义和图形化展示。
权限管理,数据权限控制增删改查执行,并且数据权限控制到行。比如DBA只能有数据库主机的权限,主机只需要一个CI模型就可以了。
基础数据,是一些系统参数。
- 协同
插件注册,选择插件包上传,并通过容器运行,支持多实例,可以查看插件运行的日志。选择插件,通过插件运行的参数关联CMDB模型的属性值,形成注册。插件包更新历史关联会自动回填。降低升级成本。目前已经有腾讯云资源管理和应用部署服务的插件。
任务编排,比如设计一个VPC创建的编排。包括创建VPC、创建子网、创建VM。流程的每个执行节点需要关联插件。
服务通道
- 设计
规划设计,用于设计机房结构。
资源规划,用于实例化一个机房,特别是两地三中心结构。
应用架构设计,用于设计一个应用的逻辑架构。
应用部署设计,用于实际部署一个应用。支持灰度发布。
CI数据管理/查询,通过模型图形进入单个数据管理以及查询。
CI综合查询管理/数据综合查询,用于配制多CI属性报表。比如一个应用使用到了哪些主机。
枚举数据管理/查询,通过对公共枚举和私有枚举进行管理以及查询。
- 执行
物料管理,管理应用程序的包。可以定义各种文件,可以配置环境差异导致的变量替换规则,不需要人工处理。
应用部署,查看部署过程及异常,并作重试处理。
批量执行,通过综合查询选择目标。在通过特定插件来执行任务。比如某个应用的所有主机,执行一个用户权限变更。
高危命令配置,定义一些高危指令(如rm -rf),阻止执行或触发审批流程。
编排任务执行,应该一般任务。比如重启某个资源集的5台主机。
- 任务(2019年下半年)
- 监测(2019年下半年)
- 调整(2019年下半年)
- 智慧(2019年下半年)
## 快速入门
WeCube采用容器化部署。
如何编译WeCube,请查看以下文档
[WeCube编译文档](wecube-wiki/docs/install/wecube_compile_guide.md)
如何安装WeCube, 请查看以下文档
[WeCube部署文档](wecube-wiki/docs/install/wecube_install_guide.md)
## 用户手册
更多关于WeCube的使用和操作说明, 请查看以下文档
[WeCube用户手册](wecube-wiki/docs/manual/wecube_user_guide.md)
## 开发者文档
WeCube使用Java和VUE进行开发,数据存储于MySQL,并依赖Tomcat Web容器运行。
请参考以下文档进行开发环境配置[WeCube开发环境配置](wecube-wiki/docs/developer/wecube_developer_guide.md)
## License
WeCube是基于 Apache License 2.0 协议, 详情请参考 [LICENSE](LICENSE)
## 社区
- 如果您想得到最快的响应,请给我们提[Issue](https://github.com/WeBankPartners/wecube-platform/issues/new/choose)或扫描下面的二维码,我们会第一时间反馈。
<div align="left">
<img src="wecube-wiki/images/wecube_qr_code.png" height="200" width="200">
</div>
- 联系我们:fintech@webank.com
# WeCube
[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
![](https://img.shields.io/github/v/tag/WeBankPartners/wecube-platform.svg)
![](https://img.shields.io/github/languages/code-size/WeBankPartners/wecube-platform.svg)
![](https://img.shields.io/badge/language-java-orange.svg)
![](https://img.shields.io/badge/language-vue-green.svg)
English / [中文](README.md)
## Introduction
WeCube is an open source, one-stop tool for architecture management and IT operation, which aims to simplify the IT management of distributed architecture. Its features can be easily extended by plugins.
## Origin
During the implementation of distributed architecture, core banking systems are built in WeBank. Comparing this to traditional monolithic architecture, there are quite a few different pain points, e.g. increasing number of servers, complicated deployment process, difficulties in service tracing and problem diagnosis due to complex systems and long invocation chains.
In the process of finding the solutions for these pain points, we have come up with a set of methodology and best practices for IT management in such scenarios and also developed necessary supporting tools. WeCube is the result of extracting the methodology and best practices form WeBank's internal tools, which is then packaged into an integrated out-of-box solution for IT management.
## Design Concept
![WeCube Design Concept](wecube-wiki/images/wecube_design_en.png)
WeCube's design concept basically matches the lifecycle of IT system management and can be elaborated as "6 aspects with 1 core".
- 1st aspect: "Workbench", it is an ITSM-like workbench that can aggregates all manual tasks so that their contents, importance and priorities are all clear at first glance.
- 2nd aspect: "Expectation", it is to plan, to design, to describe the expectations with models and specifications. By using standard and formal design language together with graphic visualizations, we try to make expectations on distributed architecture accurately and clearly.
- 3rd aspect: "Execution", it is to execute tasks in automated and standardized way so that expectations become realities without introducing errors or deviations caused by people in this process.
- 4th aspect: "Watching", it is to uncover differences between realities and expectations by defining all-round metrics which can reflect the varying runtime situation, then consistently collecting and keeping track of the information.
- 5th aspect: "Wisdom", it is to add intelligence into WeCube by means of data modeling and machine learning, so that policies can be developed to deal with errors and deviations, such as policies for trouble shooting, cause analysis, resource scaling.
- 6th aspect: "Equilibration", it is to ensure that realities are conforming to design expectations by applying policies and making consistent adjustments, so that running systems are stable in a dynamic way.
- the core: "Workflow Engine", it is to coordinate tasks and improve collaborations in those 6 aspects, so that manual intervention is reduced and team efficency is increased.
## Implementations
WeCube is composed of the core framework and plugins for feature exetnsions.
The core framework is developed with Java/MySQL/Vue.js and reponsible for workflow management, data visualization and core data persistence.
Plugins are introduced for feature extensions, they can be grouped into 3 categories: resource management, data integration and practice enhancement. By conforming to the interface specification defined by WeCube, plugin developers can choose their preferred programing languages. Existing plugins commonly use Go and Java.
## System Architecture
### WeCube Version 1.x
WeCube version 1.0 is released, its architecture is elaborated as follows:
![WeCube1.0 Architecture](wecube-wiki/images/wecube_arch_1_en.png)
- WeCube version 1.0 uses WeCMDB for data persistence, configuration data are all managed by WeCMDB.
- WeCube version 1.0 requires a CAS server for authentication, in order to keep data permissions consistent, we suggest to share the same CAS server used by WeCMDB. In WeCube version 1.1, a local user based authentication mode is supported.
- WeCube version 1.0 requires a S3-compatible object storge service for artifacts and scripts.
- WeCube version 1.0 includes 2 plugins: Tencent Cloud plugin for resource mangement and SaltStack plugin for automated deployment.
Implemented features in WeCube Version 1.0:
![WeCube1.0 Logical Architecture](wecube-wiki/images/wecube_arch_2_en.png)
### WeCube Version 2.0
WeCube Version 2.0 is under active development and will come soon.
## Main Features
The main menu of WeCube matches its design concept, including Workbench, Expectation, Execution, Watching, Wisdom, Equilibration, Workflow and System.
- System
- System parameters: All the necessary parameters used during the WeCube Platform running can be managed.
- Authorization: Users, Roles, and Menus are managed in this section, also the mapping between users, roles and menus.
- Resources: Manage resources provided by WeCube, such as container hosts and instances running on the hosts.
- Workflow
- Plugin registration: Select the plugin package to upload. The plugin package needs to declare the plugin's dependencies, required menus, data models, system parameters, authorization settings, and resources. After registration, the plugin runs through the container, multiple instances supported, and you can view the log of the plugin operation. Select the plugin service, and associate the CMDB data model attribute value with the parameters.
- Task orchestration: to design workflows to finish automated or semi-automated tasks, such as task orchestration to create VPC, subnet and virtual machines on the public cloud. Each node in the workflow is associated with a plugin for execution.
- Expectation(WeCMDB plugin)
- Data center design: to create a plan and to design the structure of data centers
- Data center resource plan: to instantiate a data center design and plan actual resources for a specific data center
- Application architecture design: to design logical architecture of applications
- Application deployment design: to design the deployment architectural view and trigger deployment for applications
- CI data query and management: to query and manage data for a single CI type through the graph of the CI model
- CI integrated query: to configure queries involving multiple CI types, for example, to find out all hosts used by a specific application
- Enumeration query and management
- Execution
- Task orchestration execution: to execute complicated operations with predefined workflow and plugins. Canary release supported.
- Artifacts(Artifacts plugin): to manage deployment packages for applications. Customization of the configuration file and value substitution rules is supported to eliminate manual intervention.
- Batch execution: to execute simple operation tasks in a selected scope of targets.
- Tasks (Service-Management plugin)
- Template management: service catalog management, service request template management.
- Service management: service request management, task management.
- Watching (Open-Monitor plugin)
- Agent management: registration, start, stop;
- Data management: Provide data collection configuration, data query, and other functions;
- Alarm management: Provides functions such as threshold configuration, log monitoring, and alarm triggering;
- View management: Provide graphic configuration and custom view functions;
- Equilibration (On Planning)
- Wisdom (On Planning)
## Quick Start
WeCube is using containerized deployment.
PLease refer to [WeCube Compilation Guide](wecube-wiki/docs/install/wecube_compile_guide_en.md) on how to compile WeCube.
PLease refer to [WeCube Installation Guide](wecube-wiki/docs/install/wecube_install_guide_en.md) on how to install WeCube.
## User Manual
Please refer to the [WeCube User Guide](wecube-wiki/docs/manual/wecube_user_guide.md) for guides on WeCube usage and operation.
## Developer Documentation
WeCube is developed with Java and Vue.js, it uses MySQL for data persistence and relies on Tomcat as the web application container.
Please refer to the [WeCube Developer Guide](wecube-wiki/docs/developer/wecube_developer_guide_en.md) on how to set up the development environment.
## License
WeCube is licensed uner the Apache License Version 2.0, please refer to [LICENSE](LICENSE) for details.
## Community
- For quick response, please [raise an issue](https://github.com/WeBankPartners/wecube-platform/issues/new/choose) to us, or you can also scan the following QR code to join our community, we will provide feedback as quickly as we can.
<div align="left">
<img src="wecube-wiki/images/wecube_qr_code.png" height="200" width="200">
</div>
- Contact us: fintech@webank.com
#!/bin/sh
mkdir -p /log
java -jar /application/platform-core.jar --server.address=0.0.0.0 --server.port=8080 \
java -Djava.security.egd=file:/dev/urandom -jar /application/platform-core.jar --server.address=0.0.0.0 --server.port=8080 \
--spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver \
--spring.datasource.url=jdbc:mysql://${MYSQL_SERVER_ADDR}:${MYSQL_SERVER_PORT}/${MYSQL_SERVER_DATABASE_NAME}?serverTimezone=Asia\/Shanghai\&characterEncoding=utf8 \
--spring.datasource.username=${MYSQL_USER_NAME} \
......
......@@ -161,10 +161,9 @@
<version>4.1.0</version>
</dependency>
<dependency>
<groupId>cn.bestwu</groupId>
<groupId>com.alibaba</groupId>
<artifactId>fastjson</artifactId>
<version>1.2-unwrap</version>
<scope>test</scope>
<version>1.2.62</version>
</dependency>
<dependency>
<groupId>com.h2database</groupId>
......
package com.webank.wecube.platform.core.controller;
import com.webank.wecube.platform.core.dto.BatchExecutionRequestDto;
import com.webank.wecube.platform.core.dto.CommonResponseDto;
import com.webank.wecube.platform.core.service.BatchExecutionService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.*;
import static com.webank.wecube.platform.core.dto.CommonResponseDto.okayWithData;
import java.io.IOException;
@RestController
@RequestMapping("/v1")
public class BatchExecutionController {
@Autowired
private BatchExecutionService batchExecutionService;
@PostMapping("/batch-execution/run")
@ResponseBody
public CommonResponseDto runBatchExecution(@RequestBody BatchExecutionRequestDto batchExecutionRequest)
throws IOException {
return okayWithData(batchExecutionService.handleBatchExecutionJob(batchExecutionRequest));
}
}
package com.webank.wecube.platform.core.controller;
import com.webank.wecube.platform.core.commons.WecubeCoreException;
import com.webank.wecube.platform.core.dto.CommonResponseDto;
import com.webank.wecube.platform.core.dto.DmeDto;
import com.webank.wecube.platform.core.dto.DmeFilterDto;
import com.webank.wecube.platform.core.service.datamodel.ExpressionServiceImpl;
import com.webank.wecube.platform.core.service.datamodel.RootlessExpressionServiceImpl;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.*;
import java.util.List;
import static com.webank.wecube.platform.core.dto.CommonResponseDto.error;
import static com.webank.wecube.platform.core.dto.CommonResponseDto.okayWithData;
@RestController
@RequestMapping("/v1")
public class DataModelController {
@Autowired
private ExpressionServiceImpl expressionServiceImpl;
@Autowired
private RootlessExpressionServiceImpl rootlessExpressionService;
@PostMapping("/data-model/dme/all-entities")
@ResponseBody
public CommonResponseDto getAllEntitiesByDme(@RequestBody DmeDto request) {
return okayWithData(expressionServiceImpl.getAllEntities(request.getDataModelExpression()));
}
@PostMapping("/data-model/dme/integrated-query")
@ResponseBody
public CommonResponseDto getAllEntitiesByDme(@RequestBody DmeFilterDto dmeFilterDto) {
List<Object> result;
try {
result = rootlessExpressionService.fetchDataWithFilter(dmeFilterDto);
} catch (WecubeCoreException ex) {
return error(ex.getMessage());
}
return okayWithData(result);
}
}
package com.webank.wecube.platform.core.controller;
import com.webank.wecube.platform.core.commons.WecubeCoreException;
import com.webank.wecube.platform.core.domain.JsonResponse;
import com.webank.wecube.platform.core.dto.PluginPackageAttributeDto;
import com.webank.wecube.platform.core.dto.PluginPackageDataModelDto;
import com.webank.wecube.platform.core.service.PluginPackageDataModelServiceImpl;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.*;
import java.util.List;
@RestController
@RequestMapping("/v1")
//@RolesAllowed({MENU_COLLABORATION_PLUGIN_MANAGEMENT})
......@@ -45,4 +49,19 @@ public class PluginPackageDataModelController {
public JsonResponse applyNewDataModel(@RequestBody PluginPackageDataModelDto dataModelDto) {
return JsonResponse.okayWithData(pluginPackageDataModelService.register(dataModelDto, true));
}
@GetMapping("/models/package/{plugin-package-name}/entity/{entity-name}/attributes")
@ResponseBody
public JsonResponse getAttributeInfoByPackageNameAndEntityName(
@PathVariable(value = "plugin-package-name") String packageName,
@PathVariable(value = "entity-name") String entityName
) {
List<PluginPackageAttributeDto> result;
try {
result = pluginPackageDataModelService.entityView(packageName, entityName);
} catch (WecubeCoreException ex) {
return JsonResponse.error(ex.getMessage());
}
return JsonResponse.okayWithData(result);
}
}
package com.webank.wecube.platform.core.domain;
import java.sql.Timestamp;
import java.text.ParseException;
import java.util.List;
import javax.persistence.*;
import com.fasterxml.jackson.annotation.JsonManagedReference;
import com.webank.wecube.platform.core.support.DomainIdBuilder;
import lombok.EqualsAndHashCode;
import lombok.ToString;
@Entity
@Table(name = "batch_execution_jobs")
public class BatchExecutionJob {
@Id
private String id;
@JsonManagedReference
@EqualsAndHashCode.Exclude
@ToString.Exclude
@OneToMany(mappedBy = "batchExecutionJob", cascade = CascadeType.ALL, orphanRemoval = true, fetch = FetchType.EAGER)
private List<ExecutionJob> jobs;
@Column
private Timestamp createTimestamp;
@Column
private String creator;
@Column
private Timestamp completeTimestamp;
@PrePersist
public void initId() throws ParseException {
this.id = DomainIdBuilder.buildDomainId(this);
}
public String getId() {
return id;
}
public void setId(String id) {
this.id = id;
}
public Timestamp getCreateTimestamp() {
return createTimestamp;
}
public void setCreateTimestamp(Timestamp createTimestamp) {
this.createTimestamp = createTimestamp;
}
public Timestamp getCompleteTimestamp() {
return completeTimestamp;
}
public void setCompleteTimestamp(Timestamp completeTimestamp) {
this.completeTimestamp = completeTimestamp;
}
public List<ExecutionJob> getJobs() {
return jobs;
}
public void setJobs(List<ExecutionJob> jobs) {
this.jobs = jobs;
}
public String getCreator() {
return creator;
}
public void setCreator(String creator) {
this.creator = creator;
}
public BatchExecutionJob(List<ExecutionJob> jobs) {
super();
this.jobs = jobs;
}
public BatchExecutionJob() {
}
}
package com.webank.wecube.platform.core.domain;
import lombok.Data;
import javax.persistence.*;
@Entity
@Data
public class BatchJob {
@Id
@GeneratedValue
private Integer id;
@Column
private String creator;
@Column(name ="script_url")
private String scriptUrl;
public BatchJob() {
this(null);
}
public BatchJob(Integer id) {
this.setId(id);
}
}
package com.webank.wecube.platform.core.domain;
import lombok.Data;
import javax.persistence.*;
@Entity
@Data
public class BatchJobHost {
public static final String BATCH_JOB_STATUS_CREATED= "created";
public static final String BATCH_JOB_STATUS_DOING = "doing";
public static final String BATCH_JOB_STATUS_DONE = "done";
public static final String BATCH_JOB_RESULT_SUCCESS = "success";
public static final String BATCH_JOB_RESULT_FAILED = "failed";
@Id
@GeneratedValue
private Integer id;
@Column(name="batch_job_id")
private Integer batchJobId;
@Column(name="host_ip")
private String hostIp;
@Column(name="output_url")
private String outputUrl;
@Column
private String status ;
@Column
private String result ;
public BatchJobHost() {
this(null);
}
public BatchJobHost(Integer id) {
this.setId(id);
}
}
package com.webank.wecube.platform.core.domain;
import java.util.List;
import javax.persistence.*;
import com.alibaba.fastjson.JSON;
import com.fasterxml.jackson.annotation.JsonBackReference;
import com.fasterxml.jackson.annotation.JsonManagedReference;
import com.webank.wecube.platform.core.support.DomainIdBuilder;
import lombok.EqualsAndHashCode;
import lombok.ToString;
@Entity
@Table(name = "execution_jobs")
public class ExecutionJob {
public static final String ERROR_CODE_SUCCESSFUL = "0";
public static final String ERROR_CODE_FAILED = "1";
@Id
@GeneratedValue
private Integer id;
@JsonBackReference
@ManyToOne
@JoinColumn(name = "batch_execution_job_id")
private BatchExecutionJob batchExecutionJob;
@JsonManagedReference
@EqualsAndHashCode.Exclude
@ToString.Exclude
@OneToMany(mappedBy = "executionJob", cascade = CascadeType.ALL, orphanRemoval = true, fetch = FetchType.EAGER)
private List<ExecutionJobParameter> parameters;
@Column
private String pluginConfigInterfaceId;
@Column
private String packageName;
@Column
private String entityName;
@Column
private String businessKey;
@Column
private String rootEntityId;
@Column
private String executeTime;
@Column
private String completeTime;
@Column
private String errorCode;
@Column
private String errorMessage;
@Column
private String returnJson;
public Integer getId() {
return id;
}
public void setId(Integer id) {
this.id = id;
}
public String getPluginConfigInterfaceId() {
return pluginConfigInterfaceId;
}
public void setPluginConfigInterfaceId(String pluginConfigInterfaceId) {
this.pluginConfigInterfaceId = pluginConfigInterfaceId;
}
public String getPackageName() {
return packageName;
}
public void setPackageName(String packageName) {
this.packageName = packageName;
}
public String getEntityName() {
return entityName;
}
public void setEntityName(String entityName) {
this.entityName = entityName;
}
public String getExecuteTime() {
return executeTime;
}
public void setExecuteTime(String executeTime) {
this.executeTime = executeTime;
}
public String getCompleteTime() {
return completeTime;
}
public void setCompleteTime(String completeTime) {
this.completeTime = completeTime;
}
public BatchExecutionJob getBatchExecutionJob() {
return batchExecutionJob;
}
public void setBatchExecutionJob(BatchExecutionJob batchExecutionJob) {
this.batchExecutionJob = batchExecutionJob;
}
public List<ExecutionJobParameter> getParameters() {
return parameters;
}
public void setParameters(List<ExecutionJobParameter> parameters) {
this.parameters = parameters;
}
public String getBusinessKey() {
return businessKey;
}
public void setBusinessKey(String businessKey) {
this.businessKey = businessKey;
}
public ExecutionJob(String rootEntityid, String pluginConfigInterfaceId, String packageName, String entityName,
String businessKey) {
super();
this.pluginConfigInterfaceId = pluginConfigInterfaceId;
this.packageName = packageName;
this.entityName = entityName;
this.businessKey = businessKey;
this.rootEntityId = rootEntityid;
}
public String getErrorCode() {
return errorCode;
}
public void setErrorCode(String errorCode) {
this.errorCode = errorCode;
}
public String getErrorMessage() {
return errorMessage;
}
public void setErrorMessage(String errorMessage) {
this.errorMessage = errorMessage;
}
public String getReturnJson() {
return returnJson;
}
public void setReturnJson(String returnJson) {
this.returnJson = returnJson;
}
public void setErrorWithMessage(String errorMessage) {
this.errorCode = ERROR_CODE_FAILED;
this.errorMessage = errorMessage;
this.returnJson = JSON.toJSONString(this);
}
@Override
public String toString() {
return "ExecutionJob [id=" + id + ", batchExecutionJob=" + batchExecutionJob + ", parameters=" + parameters
+ ", pluginConfigInterfaceId=" + pluginConfigInterfaceId + ", packageName=" + packageName
+ ", entityName=" + entityName + ", rootEntityId=" + businessKey + ", executeTime=" + executeTime
+ ", completeTime=" + completeTime + ", errorCode=" + errorCode + ", errorMessage=" + errorMessage
+ ", returnJson=" + returnJson + "]";
}
public String getRootEntityId() {
return rootEntityId;
}
public void setRootEntityId(String rootEntityId) {
this.rootEntityId = rootEntityId;
}
}
package com.webank.wecube.platform.core.domain;
import com.fasterxml.jackson.annotation.JsonBackReference;
import com.fasterxml.jackson.annotation.JsonInclude;
import com.webank.wecube.platform.core.support.DomainIdBuilder;
import lombok.EqualsAndHashCode;
import lombok.ToString;
import org.apache.commons.lang.builder.ReflectionToStringBuilder;
import java.util.Random;
import javax.persistence.*;
@Entity
@Table(name = "execution_job_parameters")
public class ExecutionJobParameter {
public static final String TYPE_INPUT = "INPUT";
public static final String TYPE_OUTPUT = "OUTPUT";
public static final String MAPPING_TYPE_NOT_AVAILABLE = "N/A";
public static final String MAPPING_TYPE_CMDB_CI_TYPE = "CMDB_CI_TYPE";
@Id
@GeneratedValue
private Integer id;
@JsonBackReference
@EqualsAndHashCode.Exclude
@ToString.Exclude
@ManyToOne
@JoinColumn(name = "execution_job_id")
private ExecutionJob executionJob;
@Column
private String name;
@Column
private String dataType;
@Column
private String mappingType;
@Column
private String mappingEntityExpression;
@Column
private String mappingSystemVariableName;
@Column
private String required;
@Column
private String value;
public ExecutionJobParameter() {
}
public Integer getId() {
return id;
}
public void setId(Integer id) {
this.id = id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String getDataType() {
return dataType;
}
public void setDataType(String dataType) {
this.dataType = dataType;
}
public String getMappingType() {
return mappingType;
}
public void setMappingType(String mappingType) {
this.mappingType = mappingType;
}
public String getMappingEntityExpression() {
return mappingEntityExpression;
}
public void setMappingEntityExpression(String mappingEntityExpression) {
this.mappingEntityExpression = mappingEntityExpression;
}
public String getMappingSystemVariableName() {
return mappingSystemVariableName;
}
public void setMappingSystemVariableName(String mappingSystemVariableName) {
this.mappingSystemVariableName = mappingSystemVariableName;
}
public String getRequired() {
return required;
}
public void setRequired(String required) {
this.required = required;
}
@Override
public String toString() {
return ReflectionToStringBuilder.toStringExclude(this, "pluginConfigInterface");
}
public ExecutionJob getExecutionJob() {
return executionJob;
}
public void setExecutionJob(ExecutionJob executionJob) {
this.executionJob = executionJob;
}
public String getValue() {
return value;
}
public void setValue(String value) {
this.value = value;
}
public static String getTypeInput() {
return TYPE_INPUT;
}
public static String getTypeOutput() {
return TYPE_OUTPUT;
}
public static String getMappingTypeNotAvailable() {
return MAPPING_TYPE_NOT_AVAILABLE;
}
public static String getMappingTypeCmdbCiType() {
return MAPPING_TYPE_CMDB_CI_TYPE;
}
public ExecutionJobParameter(String name, String dataType, String mappingType, String mappingEntityExpression,
String mappingSystemVariableName, String required, String value) {
super();
this.name = name;
this.dataType = dataType;
this.mappingType = mappingType;
this.mappingEntityExpression = mappingEntityExpression;
this.mappingSystemVariableName = mappingSystemVariableName;
this.required = required;
this.value = value;
}
}
......@@ -15,7 +15,7 @@ import javax.persistence.*;
public class PluginPackageAttribute {
@Id
private String id;
@JsonBackReference
@ManyToOne(cascade = CascadeType.ALL)
@JoinColumn(name = "entity_id")
......
......@@ -38,7 +38,7 @@ public class PluginPackageEntity {
private String description;
@JsonManagedReference
@OneToMany(mappedBy = "pluginPackageEntity", cascade = CascadeType.ALL)
@OneToMany(mappedBy = "pluginPackageEntity", cascade = CascadeType.ALL, fetch = FetchType.EAGER)
private List<PluginPackageAttribute> pluginPackageAttributeList;
public PluginPackageEntity() {
......
package com.webank.wecube.platform.core.dto;
import java.util.List;
import com.webank.wecube.platform.core.domain.plugin.PluginConfigInterface;
import com.webank.wecube.platform.core.domain.plugin.PluginPackageAttribute;
public class BatchExecutionRequestDto {
private PluginConfigInterface pluginConfigInterface;
private String packageName;
private String entityName;
private List<InputParameterDefinition> inputParameterDefinitions;
private BusinessKeyAttributeDto businessKeyAttribute;
private List<ResourceDataDto> resourceDatas;
public BatchExecutionRequestDto() {
}
public PluginConfigInterface getPluginConfigInterface() {
return pluginConfigInterface;
}
public void setPluginConfigInterface(PluginConfigInterface pluginConfigInterface) {
this.pluginConfigInterface = pluginConfigInterface;
}
public String getPackageName() {
return packageName;
}
public void setPackageName(String packageName) {
this.packageName = packageName;
}
public String getEntityName() {
return entityName;
}
public void setEntityName(String entityName) {
this.entityName = entityName;
}
public List<InputParameterDefinition> getInputParameterDefinitions() {
return inputParameterDefinitions;
}
public void setInputParameterDefinitions(List<InputParameterDefinition> inputParameterDefinitions) {
this.inputParameterDefinitions = inputParameterDefinitions;
}
public BusinessKeyAttributeDto getBusinessKeyAttribute() {
return businessKeyAttribute;
}
public void setBusinessKeyAttribute(BusinessKeyAttributeDto businessKeyAttribute) {
this.businessKeyAttribute = businessKeyAttribute;
}
public List<ResourceDataDto> getResourceDatas() {
return resourceDatas;
}
public void setResourceDatas(List<ResourceDataDto> resourceDatas) {
this.resourceDatas = resourceDatas;
}
}
package com.webank.wecube.platform.core.dto;
public class BusinessKeyAttributeDto {
private String id;
private String name;
private String description;
private String dataType;
public String getId() {
return id;
}
public void setId(String id) {
this.id = id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String getDescription() {
return description;
}
public void setDescription(String description) {
this.description = description;
}
public String getDataType() {
return dataType;
}
public void setDataType(String dataType) {
this.dataType = dataType;
}
}
package com.webank.wecube.platform.core.dto;
public class DmeDto {
private String dataModelExpression;
public String getDataModelExpression() {
return dataModelExpression;
}
public void setDataModelExpression(String dataModelExpression) {
this.dataModelExpression = dataModelExpression;
}
public DmeDto(String dataModelExpression) {
super();
this.dataModelExpression = dataModelExpression;
}
public DmeDto() {
}
}
package com.webank.wecube.platform.core.dto;
import java.util.List;
public class DmeFilterDto {
String dataModelExpression;
List<DmeLinkFilterDto> filters;
public DmeFilterDto() {
}
public DmeFilterDto(String dataModelExpression, List<DmeLinkFilterDto> filters) {
this.dataModelExpression = dataModelExpression;
this.filters = filters;
}
public String getDataModelExpression() {
return dataModelExpression;
}
public void setDataModelExpression(String dataModelExpression) {
this.dataModelExpression = dataModelExpression;
}
public List<DmeLinkFilterDto> getFilters() {
return filters;
}
public void setFilters(List<DmeLinkFilterDto> filters) {
this.filters = filters;
}
}
package com.webank.wecube.platform.core.dto;
import java.util.ArrayList;
import java.util.List;
public class DmeLinkFilterDto {
int index;
String packageName;
String entityName;
List<Filter> attributeFilters;
public DmeLinkFilterDto() {
this.attributeFilters = new ArrayList<>();
}
public DmeLinkFilterDto(int index, String packageName, String entityName, List<Filter> attributeFilters) {
this.index = index;
this.packageName = packageName;
this.entityName = entityName;
this.attributeFilters = attributeFilters;
}
public int getIndex() {
return index;
}
public void setIndex(int index) {
this.index = index;
}
public String getPackageName() {
return packageName;
}
public void setPackageName(String packageName) {
this.packageName = packageName;
}
public String getEntityName() {
return entityName;
}
public void setEntityName(String entityName) {
this.entityName = entityName;
}
public List<Filter> getAttributeFilters() {
return attributeFilters;
}
public void setAttributeFilters(List<Filter> attributeFilters) {
this.attributeFilters = attributeFilters;
}
}
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment