1. Introduction The Data Mover Challenge 2021/22 is a challenge organized by NSCC Singapore since 2018. The charter of the DMC is to build a platform and bring together experts from industry and academia to test their high performance data transfer solutions across the Data Transfer Nodes testbed set up in various countries connected by 100G and beyond international networks. The challenge focuses on optimizing point-to-point data transfers between sites – a crucial step forward in advancing research collaboration and sharing. Participants from all over the world will compete by deploying the best software tools on Data Transfer Nodes that are set up within existing international networks across the globe. 2. The Challenge DMC 2021 will run from 1 August to 31 October 2021. Participating teams will be given 5 days to deploy and run data transfer tests based on given scenarios: - 3 days to set up the software - 2 days to run the data transfer and demonstrate software to judges A video conference zoom session will be set up for participants to present their solution and results with interview from the judges prior to the data transfer demonstration test. 3. Network Topology 4. Data Mover Challenge Competition Schedule 7 teams will take part in the competition. And Raysync will run the data transfer tests from 27th Sep to 1st Oct. The winning team’s leader will be invited to attend SupercomputingAsia 2022 https://www.scasia.org/ in Mar 2021 in Singapore for an award presentation and solution showcase. 5. About Raysync Solution Basing upon UDP’s high-speed transfer protocol, Raysync breaks through the bottleneck of large file transfer which satisfies enterprises’ needs by building an intelligent transfer platform that processes and integrates massive data at an unprecedented speed. Our solutions enable you to send files of any size or format at full line speed, hundreds of times faster than FTP, while ensuring secure and reliable delivery. For more info and pricing packages, please visit www.raysync.io.

Introduction A peer-to-peer connection in telecommunications relates to a communication between two terminals or nodes. Peer-to-peer is shortened to P2P sometimes. In sharing platforms or in other data-sharing protocols between people, this use of P2P is separate from the use of P2P which means peer-to-peer. For instance, a telephone call when one phone is connected to another, and only the others could hear what one caller says. In contrast, this is a topology for point-to-multipoint or broadcast communication, where several nodes can receive information from a node. Additional instances of peer-to-peer links include leased lines, microwave relays, and two-way radio. For example, radio and television broadcasts are points-to-multiple. Have you used P2P? P2P transfer is now widely used to share files by many enterprises in their work after LAN, FTP, and HTTP are outdated. P2P transfer, known as peer-to-peer transfer, is an instant file transfer mode, which aimed to enable all clients to provide resources, including bandwidth, storage space, and computing capacity. P2P transfer technology has wide application scenarios, such as audio, video, and data sharing in various formats. How does P2P technology transfer files instantly? Why P2P transfer is accepted widely by many enterprises? In this article, let’s start with these two questions and explore the answers to peer-to-peer transfer. Everything about P2P Transfer in One Picture With the growing number of IP cores combined on one chip, the system-on-chip architecture is able to deploy complicated applications. The enormous need for communication and the ample computing capability on the chip put a considerable burden on the architecture in communication. Scalable communication architectures are therefore needed to implement future systems efficiently. Two types of communication systems, namely peer-to-peer and bus-based designs have traditionally been explored. At the expense of the specific channels between all communicating IP pairs, P2P communication designs can provide the most communication performance. However, in terms of high complexity, expense, and design effort these designs are poorly scalable. By contrast, bus systems can connect a few dozen IP cores economically and eliminate the complexity of the system and the specific cables required by P2P systems. The traditional HTTP transfer is to copy data to the client via server, thus the transfer speed is restricted by the bandwidth. Especially in today's network environment, the large amount of clients connects to one server, the bandwidth of servers will be shared, and the bandwidth of clients is difficult to be fully utilized, resulting in low efficiency of overall data transfer. In general data transfer, client A and client B is essential to build a connection to the server, and then data will be transfer through the channel built by the server and data recipient. P2P transfer technology builds the direct channel between client A and client B, shorten the original upload-download "1+1" time to 1; Secondly, "client A-server-client b" is turned to "client A-client b", which does not transit through the server and does not consume the expensive storage space of the server. Figure 1. P2P Transfer in One Picture Cons & Pros of P2P Transfer P2P technology is popular in many industries such as film and media, Internet TV, telemedicine, it not only provides freedom and facility to data transfer but also integrates all potential Internet resources. In technical expert’s opinion, P2P transfer is a kind of instant technology for quick data moving and sharing, it is beneficial to creating opportunities for enterprises in today’s market environment driven by data. However, with 5G emerging and the continuous development of information technology, P2P transfer is undoubtedly a double-blade sword, so it’s difficult for enterprises to step forward. Pros: Low cost: save the cost on the server. Efficiency improved: remove the intermediate procedures and establish simple connections. High stability: eliminate the server-centered single-point service in the past, form the decentralized P2P transfer organization, which solved the fault problem caused by the central single point. Cons: Network bandwidth resources are consumed in large quantities: The concurrent connection of P2P transfer makes the resources consumed in large quantities, it is easy to cause network congestion and reduce the performance of other applications. Inconvenient management: The P2P transfer is decentralized, the amount of users is huge thus it’s hard to take care of all users. Security risks of data leakage: The widely use of P2P transfer leads to someone use the loopholes in the system to invade user privacy and destroy data. Some criminals even add viruses in the P2P transfer process, which poses a threat to users' computer security. The P2P transfer technology is not strong enough to satisfy enterprises’ various needs, therefore, there must be a breakthrough. According to whether the clients are in the same network environment, the difficulty degree of P2P transfer technology is different. What modern technologies can be used to optimize and how to optimize? Let's continue to explore the principle of P2P technology and try to find a breakthrough. P2P technology upgrading Based on NAT+Raysync The transmission protocol of Raysync has abandoned the use of transmission messages as both byte counts and TCP protocol. With its revolutionary UDP transmission method, Raysync overcomes faults in existing TCP-based file transfer protocols . The transmission speed of FTP/HTTP is hundreds of times higher and is not reduced by the size of the file, transmit distance, or network difficulties. Raysync can offer secure internal and external sharing of data, enabling you to securely upload, download and edit files on a user-friendly interface. Send huge files via mobile devices and web browsers in every location and anywhere. Characteristics Raysync transmission protocol may make full use of a broad network of bandwidths for the fastest transfer of data. It can enable users in the huge data distribution application scenario to finish the big data transfer in the least time. The multi-channel parallel transmission feature of the link is supported by RaySync Transmission Protocol. When a client interfaces between point B and point A, a multiplicity of unrelated data streams can quickly be transferred in Parallel via the multi-parallel channel property. The transfer of these data streams does not require the handshake connection process, which considerably enhances the transmission in real-time. Raysync transmission protocol is UDP-based and may transit over various NAT devices smoothly. Here we divided into two situations for analysis: On the same network Assume that the internal network interface is 1000M, the LAN bandwidth is 10M, and the bandwidth utilization rate is 50%. Client A and Client B are in the same network environment, as shown in the following figure. If the traditional file transfer method is adopted, the A user needs to upload the file to the server first, and then B can download it from the server, there only 10M network bandwidth will be utilized in the whole transfer process. While through peer-to-point transfer, A and B can directly establish a connection, file transfer makes full use of the network bandwidth of 1000M, and the transfer efficiency can be improved by 100X. Figure 2. P2P technology on the same network Raysync combined the P2P technology with Raysync ultra-high-speed transfer protocol, the network bandwidth is fully utilized up to 96%. The file transfer process is accelerated, the TB-level big data and massive data transfer have never been easier. Figure 3. P2P technology upgrading Based on NAT+Raysync On the different network Client A and client B are in two different network environments, and they need to try to connect with the target end to judge whether they can penetrate the network to establish communication. The P2P transfer between two clients will be different according to different clients. Several common P2P transfer methods through middleware include Relaying, Connection reversal, UDP hole punching. Through the following characteristic comparison table, we can simply grasp the characteristics and application scenarios of these methods. Table 1. Characteristics and compassion table According to the methods listed above, Raysync P2P transfer will try to connect with the recipient end in priority. If it is failed to connect, it will be connected by reverse connection or UDP punching. If it is still failed to connect, then the server transfers will be adopted. Raysync will automatically select the best transfer mode in the whole process, reduce the network burden and make full use of network resources. Besides, the P2P transfer based on UDP needs to consider the type of NAT, because the penetration ways of different NAT combinations are not consistent, some can penetrate, while some cannot. Figure 4. P2P technology on the different networks Digging deep into NAT penetration technology, Raysync combines more than 10 NAT penetration technologies with Raysyncb ultra-high-speed transfer protocol and simultaneously applies on P2P transfer. In practical application, the penetration effect of these 10 NAT combinations can meet the current P2P transfer needs. Even in the face of a few impenetrable situations, Raysync may still be able to improve the transfer efficiency and find the optimal solution for P2P transfer. A More Universal P2P Transfer through Innovation on Speed and Management Transfer acceleration ensures efficiency improving In the peer-to-peer transfer, the patented Raysync ultra-high-speed transfer protocol is adopted. This protocol can eliminate the bottleneck at the bottom layer, it is immune to limitation of the traditional network and hardware, fully utilize the network bandwidth, and realize ultra-low latency, high speed, and end-to-end output service, the transfer rate is improved by nearly one hundred times, thus easily achieves the safe, controllable and stable transfer of TB-level large files and massive small files. Figure 5. P2P technology on the different networks about penetration and acceleration Comparison of the peer-to-peer transfer efficiency Test environment: Shenzhen-Beijing, network latency 35ms~45ms, packet loss rate 1% Server: Alibaba cloud Shenzhen, CentOS 8.3, 2 vCPU 4 GiB, bandwidth 100M, SSD disk ClientA: Alibaba cloud Shenzhen, CentOS 8.3, 2 vCPU 4 GiB, bandwidth 100M, SSD disk ClientB: Alibaba cloud Beijing, CentOS 8.3, 2 vCPU 4 GiB, bandwidth 100M, SSD disk File transfer is automatic For team collaboration, Raysync supports automatically receive peer-to-peer files from your partners. Compared to the traditional P2P transfer method, Raysync saves your time spend on manually file reception. The auto file reception can be realized with one click to enable the “Allow to receive transfer files from partners” button. A data asset is strictly monitored The admin user can monitor every single task in the admin console. Enter Raysync admin console – peer-to-peer transfer, you could view the peer-to-peer transfer account, IP, task name, receiver ID, country, states, city, total file transmission quantity, and transmission speed. In the meantime, it supports viewing the details of the peer-to-peer transfer task being transmitted and stopping the operation, and supports one-click viewing and downloading for historical transfer tasks. All files and individuals traceable, each P2P task can be controlled by the admin. The comprehensive upgrade made by Raysync on P2P transfer overcomes the drawbacks of file and user management, enables peer-to-peer transfer to better meet the enterprise needs, and helps enterprises manage data more efficiently. Bank-level data security Based on the needs of customers in industries such as film and media, IT Internet, Raysync adopts AES-256+TLS encryption technology between client and server to guarantee end-to-end data security. Aiming at the port problem, Raysync only needs to expose one port, which can meet all users' access needs and greatly reduce the risk of firewall port exposure; Hash verification protection is carried out at the transfer message level, file block, and the whole file to ensure the integrity of the transfer content. The optimization made by Raysync on peer-to-peer transfer, which has outstanding performance in performance, reliability, and security. The upgraded management design meets the needs of modern enterprises. As a one-stop solution provider for large file transfer, Raysync can be used as a tool for enterprises to realize fast file cooperation, and can also be used as an enterprise data management platform to provide data transfer services for industries such as film and media, finance, IT Internet, etc.

At work, we almost certainly encounter problems of file synchronization, and if there are many files modified every time, the manual comparison is a very troublesome thing, so we need assistance from the file synchronization tools, especially two-way file synchronization tools. What is the best file synchronization software? 1. Raysync Raysync file synchronization function, through the core technologies independently researched and developed, such as high-speed engine technology and high-speed transfer protocol technology, establishes a data synchronization solution in line with the business scenarios of enterprise users. 2. DiffSync DiffSync is an expert in file synchronization. It is powerful and can compare files according to their size, creation time, modification time, access time, file attributes, etc., and synchronize differences according to the comparison results. Unique high-performance algorithm makes file comparison so fast. Ultra-fast file copying makes it easy to synchronize a large number of files. 3. FastCopy FastCopy is the fastest file copying software on the Windows platform. It supports UNICODE and file path names exceeding MAX_PATH , developed by a Japanese company, and supports simple file synchronization function. 4. FreeFileSync FreeFileSync is a folder comparison and synchronization software, which can create and manage backup copies of all important files. FreeFileSync does not need to copy every file every time but determines the difference between the source folder and the destination folder and transfers only the minimum amount of data required. FreeFileSync is open source software, which can be used for Windows, macOS, and Linux. 5. GoodSync GoodSync is a simple and reliable software for file backup and file synchronization. It automatically analyzes, synchronizes, and backs up your e-mail, precious family photos, contacts, MP3 songs, financial files, and other important files, such as desktops, laptops, servers, external drives, Windows Mobile devices, as well as WebDAV of users remotely via FTP, etc. 6. Allway Sync Allway Sync is a Windows file synchronization software. It can synchronize files among several folders and automatically overwrite the updated files with the old files in several synchronized folders. Allway Sync is such a free and easy-to-use Windows file synchronization software, which can synchronize several different folders in the computer with each other, and also with folders such as network neighbors, FTP, WebDAV, mobile hard disk, etc., with powerful functions. It supports XP~Win10 system and Chinese. 7. PathSync PathSync is a directory file synchronization tool. PathSync can analyze two directories and display a list of differences between directories to users. Users can select the actions to be performed and allow PathSync to synchronize.

File Transfer Protocol Server is a computer that provides file storage and access services on the Internet. They provide services in accordance with the FTP protocol. FTP, the file transfer protocol is a set of standard protocols for file transfer on the network, using the client/server model. FTP is a protocol specifically used to transfer files. FTP is a client/server system. Users use a client that supports the File FTP Protocol connect to a server program on a remote host. The user issues a command on the client, and the remote host server executes the command issued by the user after receiving the command, and returns the execution result to the client. Simply put, the user sends a command to the server, asking the server to send a file to the user. The server responds and sends the file to the client. The user receives the file and places it in the user's working directory. This process is carried out by the FTP server. How Does FTP Server Work? FTP is based on the TCP protocol service, which is a file transfer protocol on the Internet. By default, two ports 20 and 21 are used, a data port and a command port, and port 20 is a data port for file transfer between the client and the server data flow. Port 21 is a command port used to transmit control flow and accept related FTP commands and parameters issued by the client. The FTP client sends a service request to the FTP server in the computer network. The FTP server receives and responds to the FTP client's request and provides the FTP client with the required file transfer service. According to the provisions of the TCP protocol, FTP servers use well-known port numbers 20 and 21 to provide services, and FTP clients use temporary port numbers to send requests. The FTP protocol specifies different well-known port numbers for control connections and data connections, the well-known port number for control connections is 21, and the well-known port number for data connections is 20. The FTP protocol uses a continuous connection communication method, and the maintenance time of the control connection it establishes is usually long. After explaining the main connection method of the working principle of FTP, then explain the two types of data connection establishment in the working principle of FTP: active mode and passive mode. The active mode of FTP is that the client connects to the well-known port of the FTP server from any unprivileged port, that is, port 21. After receiving the command, the FTP server connects to another temporary port of the client from the data port and returns data; passive mode means that the client opens two arbitrary non-privileged ports to submit commands, and the FTP server passively opens any non-privileged ports to send commands to the client. After the client receives the command, it initiates a connection to the FTP server from the local port to establish a transmission data channel. In this mode, the command connection and the data connection are initiated by the client to solve the problem of the server initiating the connection to the client. Generally speaking, if we take the server as the criterion. Active mode: The server initiates a connection to the client from port 20. Passive mode: The server passively waits for a client to initiate a connection on a port within a specified range. The realization goal of FTP is to realize the safe sharing of file data, that is, the computer client can connect to the FTP servers around the world through the FTP client and the Internet to achieve file data sharing and resource sharing. Realize information interaction, that is, other computer clients can access the FTP server built on your computer through the Internet, so that your information can be spread to all parts of the world. Carry out effective data transmission. Files can be exchanged between computers of different types, different systems, and different formats. Do you know the FTP server now? For related articles, you can click here to learn more.

"Raysync Industry Solution" is a summary of the service experience of Raysync to 2W+ enterprises, covering the mature solutions of ten major industries such as film and television media, manufacturing, government agencies, IT Internet, automobile, and finance. Today, we make an in-depth analysis report on the IT Internet industry. Challenges: 1. Low data transfer efficiency Under the original office mode of enterprises, departments are independent of each other, data sharing and transfer are not timely, and communication costs are high. At the same time, data movement efficiency lags behind due to the distance between headquarters and local branches and the limitation of the network environment. 2. Busy on cloud storage It is a general trend to store data in the cloud. However, due to the transfer speed of the local network and cloud network, the time for enterprises to go to the cloud often becomes a problem. 3. There are many projects and complex collaboration Due to the complexity of projects in the Internet industry, there are many parallel or even cross projects. Communication and cooperation between projects and traditional transfer management methods are eas y to cause version confusion, data overlap, and inefficient cooperation delays project progress. 4. Data security guarantee As one of the core tasks of IT enterprises, data security is related to enterprise development. How to prevent potential risks such as data leakage, data damage or loss, and personnel flow, and ensure the safety of important digital assets of enterprises. Solutions: - According to the organizational structure of the enterprise, Rasysync quickly establishes transfer channels, opens up information sharing channels of all parties, and effectively realizes the accelerated transfer of inter-departmental data. - Support setting corresponding personnel authority, realize centralized management of enterprise documents and precipitation of knowledge and maximize the value of data. - The whole transfer process is encrypted by SSL, which ensures the security of data transfer, does not need to worry about malicious theft of core data by a third party in the transfer process, and supports specific operations such as the user-defined setting of use times, validity period and use the time to ensure the safety of data outgoing. Click to get the complete Raysync Industry Solutions Raysync file transfer solution is dedicated to meet the data transfer requirements of internal or external partners of enterprises, providing efficient and controllable accelerated transfer of large files, ultra-long distance and transnational network data transfer, safe outgoing of file assets, file management and organization authority management, supporting local deployment and cloud services, and providing enterprises with safe, stable, efficient and convenient technical support and services for large file interaction.

Facing the wide variety of office application products in the market, how to select the best products is an inspection of the product capabilities, and also an inspection of the capabilities of an enterprise. As an enterprise-level large file transfer software, during the communication with enterprise customers, it is found that there are two key points that are easily overlooked when most enterprises choose file transfer software. Deployment conditions is nescessary The feature of ease of use is the basis to select a file transfer software, an excellent file transfer software should make users comfortable from deployment to use. In the file transfer software, Raysync saves users' time and is convenient to operate. It can be accessed by the WEB browser and used after 30 seconds of installation. For more deta ils about deployment, you can visit raysync.io -> Support to learn more. Security Protection can be neglected For file transfer software, good software should not only ensure the security of the file transfer process but also ensure the security of static data. Raysync adopts TLS+AES-256 encryption algorithm technology, so that data can reach international financial security. In file distribution, central control is supported, data can be encrypted, file access supports identity authentication, and file-sharing can be managed in groups. Let the enterprise know every document like the back of its hand, and keep every data under control.

When making video content in 2020, people not only pay attention to the content quality but also pursue the best video experience. In view of this, when making a video plan, the film and television production team needs to determine the size of the created video files and how to quickly transmit them to the team and audience. The purpose of this article is to provide the creators of the media industry with new expanding information. We will analyze the reasons for the continuous growth of video file size and the available solutions to manage this growth without losing video quality and data. Film Length When creating a storyboard for video content, one of the first things to consider is the length of the video file. As this is the framework of the content, it plays an important role in determining the file size. Obviously, the shorter the duration, the smaller the video size can be estimated. Video Resolution Before shooting begins, we need to obtain specific tools to make the content reach the desired quality level, so we may need to buy specific cameras, editing tools and transfer software. At the same time, video quality will ultimately affect the size of your video file through pixels. The higher the video quality, the more pixels that make up the image and the larger the file size. Larger files will make it more difficult to transfer files. At this time, it is particularly important to have available tools to provide you with multiple resolution options. Raysync large file transfer solution can work without limiting video material flow, and has quality assurance for large file transfer and transnational file transfer. Video Format Video format has two important components: codec and container. Codec refers to the process of how to compress files, and container refers to how to save video files together and display the files by the types of file extensions . In format conversion, a two-hour 8K video is produced most often, and the file must be compressed to output it as a smaller video file. In the process of compressing large video files, the possibility of data loss is very high. High-quality compression solution is very beneficial to protect video quality and shorten the time of high-speed file transfer. Video Frame Rate A quick way to know the video size in use is to know the frames per second . The higher the FPS, the better the video quality. Whether it is a film and television production company or a start-up team, a team is always needed to make video files. There are three factors that you need to know about how to transfer such large files quickly between teams. - Distance: your data transfer distance - Delay: refers to the delay in transmitting or processing data - Bandwidth: the maximum transfer rate of a network or Internet connection, and how much can be transferred in a specific time The fact is, our technology and life are constantly improving, and the image resolution will continue to improve. Because it is a trend and a product of the progress of the times, we need to create new tools to accommodate oversized video files. Raysync provides efficient and safe large file transfer solutions to solve the problem of large file transfer for media industry workers.

Company: Studio 51 Industry: Motion Pictures & Film Product: Raysync SMB Profile: As a visual effects production studio founded in 2013, Studio 51 is dedicated to film, entertainment and advertising industries. Collaborating with the local and global network of experienced talented artists, Studio 51 can provide a broad range of creative solutions. They offer professional services including concept design, pre-production, shoot supervision, and post-production. Challenges: The challenge Studio 51 faced is to exchange data around the world. Because of the inherent problems associated with transferring large files abroad via traditional methods, such as latency and packet loss, the transfer time was unacceptably slow, resulting in avoidable downtime and overall workflow inefficiency. Solutions: To overcome the challenges, Studio 51 deployed Raysync SMB. The Raysync admin console is deployed in Hong Kong, then the fast and secure data exchanging is guaranteed between Studio 51’s other offices in Germany, Sweden, and India through Raysync high-speed transfer protocol. Raysync SMB provides Studio 51 with fast file transfer speeds and maximizes their bandwidth and guarantees that all of the content that distribute or receive arrives on time. Besides, Raysync helps Studio 51 to create multiple user groups and give them different access permission, which simplified the work procedure, made the management easier, and greatly improved the work efficiency. Results: Raysync high speed file transfer solutions enabled The Studio 51 team to edit and transfer all of their content from across the globe at blazing speeds. With the enhanced methods from Raysync SMB, the Studio 51 was able to achieve speeds hundreds of times faster than traditional methods, ensuring the delivery of large files from anywhere in the world, while providing security, reliability. Workflow: Raysync & Studio 51 Conclusion We were fortunate to work with Studio 51, and the cooperation and communication between the two parties was very smooth during the cooperation. Through this cooperation, Raysync has a deeper understanding of file transfer in the film industry and will continue to improve the transfer technology in the future. Once again, we wish studio 51 the best of luck!