DynFi and AR24
DynFi deploys a large CEPH cluster for AR24 using the latest SDN technologies from Proxmox.
AR24, a subsidiary of La Poste (French Post Office) specialized in online registered letters, wanted to modernize its hosting infrastructure initially based on Apache Cloud Stack. CloudStack technologies were too complex and difficult to scale.
After carefully reviewing and analyzing existing technologies, AR24’s engineering team decided to select a Proxmox VE cluster solution based on CEPH, incorporating the latest SDN technologies embedded in version 7 of PVE. The team at DynFi helped AR24 define and implement the architecture with 100% open source technologies.
“The Proxmox VE and SDN technologies allow us to have a highly resilient unified service stack with a CEPH custer as the central storage point. This, without making any compromise on Cybersecurity” reports Hubert Hardy, CTO of AR24. “The criticality of our customers’ data is the main reason for choosing the Proxmox VE solution. The recent arrival of the Proxmox Backup Server system reinforced our choice, as did the ability to provide BGP routing with the BGP/L2VPN-EVPN gateway recently embedded in PVE.”
Data security with CEPH and CephFS
One of the challenges for the project was to have a powerful cluster offering optimal performance for the VMs while having a large storage space for the data and an important “scalability” capacity.
This challenge was met by using CephFS and CEPH within the cluster. The use of NVMe and SSD disks allowed us to optimize the overall storage while ensuring high performance for disk access.
The use of Proxmox Backup Server ensured an excellent backup for the data from RBD. Other datas stored in CephFS file system were backed up using another technology that was more efficient for this use case.
DynFi was thus able to test numerous scenarios and optimize the implementation of AR24’s backup policy. Some of the tests we conducted were the subject of an advanced benchmark, results of which are available online.
Deploying a Cluster compatible with advanced BGP routing
Proxmox VE since version 7 allows the deployment of an SDN stack that can be configured using several different implementation formats (VLAN, QinQ, VXLAN, BGP-EVPN).
To meet AR24’s needs, it was necessary to have a perfect compatibility with the BGP protocol and thus to be able to have an end-to-end L2VPN-EVPN routing up to the VMs and this whatever their position on the Proxmox cluster (independently of the node of the cluster where the VMs are actually deployed at any time).
The solution to solve this problem was to use the L2VPN-EVPN brick available in Proxmox VE.
For design reasons and in order to meet our commitment to be 100% open source, we have decided to use the FRR routing software. This software has allowed us to simplify the implementation of FRR by ensuring a good level of segmentation between eBGP and iBGP. We preferred to use a native Linux implementation rather than the one proposed in some switches on the market.
No compromise on Cybersecurity
The initial design of the solution has allowed us to take into account Cybersecurity issues at a very early stage.
Different levels of firewalling are implemented, including at the PVE cluster level. The use of the PVE firewall allows, thanks to the use of macro, to simplify the deployment of associated rules.
Finally, the flows are distinctly routed in order to offer a good segmentation facilitating the global analysis of the traffic and improving the cybersecurity of the solution.
Some other types of firewalling solutions are deployed at various level in order to meet the highest level of cybersecurity.
Conclusion: Proxmox VE the solution for your hosted clusters
By using Proxmox VE, AR24 was able to create a complete and extremely rich housing infrastructure compatible with the latest L2VPN-EVPN + BGP technologies.
This infrastructure meets the requirements of tier-1 or tier-2 operators and offers the possibility of deploying several operators in multi-homing.
The last challenge will be to make multi-site based on BGP with the possibility of having an automatic recovery in case of failure of one of the site.
Contact
- DynFi
- Country : France
