About DBStream
DBStream supports all editions of SQL Server 2008 R2, SQL Server 2012, SQL Server 2014, SQL Server 2016, SQL Server 2017, and SQL Server 2019. It's currently distributed as a C++ console application. DBStream has been designed for almost all the environments, wherever the source and the target are. DBStream should NOT be considered as the replacement of traditional database backup solution. DBStream is NOT open source or free.
We're pround of DBStream, for it's
1. Super Fast - It's a HUGE improvement that backup and restore can run in parallel. The performance almost depends on the network between source and target.
2.
Super Easy - Just two commands. Then data streams to target from source block by block via TCP connection as 'data pipeline' . No worry about storage, FTP, share, S3, firewall, etc.
3. Super Safe - There is NO backup file, but small data pieces in memory as cache. It significantly eliminate the concerns on backup file security.
4. Low Costs - No need of intermediate storage for backup file. And it dramatically reduces the huge potential communication costs and 'un-necessary' efforts between different teams.
And more ...
DBStream is derived from GABR. Functionally, DBStream is a small subnet of GABR. GABR is relatively much more complicated backup & restore solution with a lot of others features, such as Compression, Encryption, AWS S3 support, etc. But, in the practice, considering,
1. Build-in Compression & Encryption become better and better in new versions of MS SQL Server. It has been able to meet the requirements in most of scenarios.
2. Many 3rd party tools are already available in market with the similar features including Compression, Encryption, AWS S3 support, etc.
3. DBAs need something innovative, such as the endpoint-to-endpoint backup & restore, to face the specific challenges.
4. GABR is too complicated. In many cases, DBA just need a super easy way to backup and restore.
we extracted endpoint-to-endpoint backup and restore feature out of GABR as DBStream with tons of consolidations.
Typical Cases, Pains & Concerns
It's common requests for DBA to backup databases in one server, and restore them in other servers. It sounds pretty easy. Actually, it may not be as easy as expected in the real world. Here is some typical cases.
- CASE 1 - In the same datacenter, backup some databases in PROD environment, and restore them in UAT environment
Step Description Operation Concerns Security Concerns 1 Backup databases in source to the share or some local drive 1. Enough space available?
2. Impacts on storeage subsystem to others?1. Any sensitive date in backup?
2. Are the backup files accessible to any unauthorized person?
3. Are the backup files deleted in time after restore?2 Copy backup to the place where target can access 1. Enough space available?
2. File Share is open? Any firewall issue to copy files between different ENVs?3 Restore databases in target - CASE 2 - Backup some databases in primary datacenter, and restore them in secondary(DR) datacenter
Step Description Operation Concerns Security Concerns 1 Backup databases in source to the share or some local drive 1. Enough space available?
2. Impacts on storeage subsystem to others?1. Any sensitive date in backup?
2. Are the backup files accessible to any unauthorized person?
3. Are the backup files deleted in time after restore?2 Transfer backup to DR site where target can access 1. Is there enough space available?
2. File Share is open? Any firewall issue to copy files between different datacenters?
3. Is it fast enough?3 Restore databases in target - CASE 3 - Backup some databases in on-prem server, and restore them in AWS EC2 instances
Step Description Operation Concerns Security Concerns 1 Backup databases in source to the share or some local drive 1. Enough space available?
2. Impacts on storeage subsystem to others?1. Any sensitive date in backup?
2. Are the backup files accessible to any unauthorized person in all places such as share, AWS S3, etc?
3. Are the backup files deleted in time after restore?2 Upload backup to AWS S3 1. Any firewall issue to upload backup to AWS S3?
2. Is it the backup too big?
3. AWS credential may expire in the middle?
3. Fast enough?3 Download backup from AWS S3 to the place where target can access 1. Is there enough space available?
2. Does it introduce performance issue to current sessions?4 Restore databases in target - CASE 4 - Backup some databases, share with external clients who need to restore them in their own database server
Step Description Operation Concerns Security Concerns 1 Backup databases in source to the share or some local drive 1. Enough space available?
2. Impacts on storeage subsystem to others?1. Any sensitive date in backup?
2. Are the backup files accessible to any unauthorized person in all places such as share, FTP, etc?
3. Are the backup files deleted in time after restore?2 Move backup to FTP site 1. Any firewall issue to move backup to FTP site?
2. Enough space available?3 Download backup from AWS S3 to the place where target can access 1. Enough space available?
2. Fast enough?4 Restore databases in target
If there is any firewall issue, space issue, performance issue or security concern, back-and-forth communctions and much efforts across different teams look inevitable. It's absolutely expensive and boring, and full of pains. And the slowness of backup copy between different locations makes it even worse.
Dream & Efforts
Oilbergs are filled in mid-east, carefully escape from bad weather, pirates and submerged reefs, then cross narrow channel & canal, and finally arrives at the destination with tons of efforts. Oil pipeline looks like a perfect solution.
About database backup-and-restore, the substantial is to move data from source to target. So why NOT stream it like the way of oil pipeline? After all, the network (pipeline) is cheap, fast and stable, and it becomes even better and better, as time goes by.
A dream of DBA on database backup-and-restore may be as easy as below.
1. Target said - I'm ready to restore database
2. Source - OK, here you are
3. Then database streams from source to target
DBStream's efforts are exactly for achieving this. To backup a database in source and restore it in target, we only need two commands.
1. In target - run one command to start Restore service
2. In source - run the other command to backup and stream the database
Then, the data will flow from source to target block by block via TCP connections.