-
Notifications
You must be signed in to change notification settings - Fork 0
/
NEWS
134 lines (107 loc) · 5.2 KB
/
NEWS
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
Release Notes
=============
Release 0.1 7/5/2011
--------------------
Committer: David Lapsley<[email protected]>
First internal release. End to end data path, supporting infrastructure.
- Incorporated PF_RING code and module to get 8 Gbps NIC to Drive throughput.
- Implemented continuous integration and unit testing framework.
- Main program with options parsing.
Release 0.2 7/7/2011
--------------------
Committer: David Lapsley<[email protected]>
Major performance milestone.
Using a a single 10 GE network interface, and a single disk pack
(with only 7 disks, one has failed), able to get ~ 7.6 Gbps net
to disk throughput (with 3% packet loss and ~95% utilization of a
single CPU ~ out of 16 total CPUs). Throughput on both transmit
and receive checked, and volume of data written over test interval
checked. Test duraiton was 30 seconds.
- Ported code to debian squeeze (6.0.2) with 2.6.32 kernel.
- Removed source dependencies in favor of binary debian packages.
- Incorporated PF_RING module source into build tree.
- Upgraded "setup" build script. Includes more user accessible
functionality. Also automatically installs all packages required
to create a build/development machine from scratch.
- net2disk module now fully based on PF_RING architecture.
- Added tools subdirectory with network and disk configuration scripts.
Disk configuration scripts include optimizations that disable
journaling on ext4 file systems and enable driver level throughput
optimizations.
Release 0.4 7/15/2011
---------------------
Committer: David Lapsley<[email protected]>
Major performance milestone. Application software achieves maximum
network to disk throughput supported by hardware platform and drivers:
14.1 Gbps sustained for 60 seconds (still room to optimize platform
configuration). This was using 4 x 8-disk modules. Each module
configured as a separate RAID-0 array.
- Finalized application architecture (multi-process architecture
rather than mult-threaded)
- Optimized PF_RING driver parameters
- Optimized Myricom driver settings
- Optimized IRQ SMP processor affinity for LSI MegaRAID and
Myricom Ethernet devices.
- Optimized application process SMP processor affinity.
- Optimized application buffer sizes, and data path.
- Optimized network packet capture code.
- Optimized internal buffering code.
- Optimized disk writing code.
- Application includes the following major classes: NetReader, BufferPool,
FileWriter, and StatsWriter.
- NetReader: Reads data from PF_RING buffers and fills BufferPool allocated buffers with the data.
- BufferPool: Internal buffers optimally sized and aligned for Direct Disk IO. Includes optimized memory management.
- FileWriter: Manages memory to disk data transfer.
- StatsWriter: Instrumentation class. Periodically logs network and disk performance statistics.
- Interactive command line interface that allows user to launch and
stop capture threads in a convenient manner.
- Startup scripts that automatically set MegaRAID, Myricom, and
Application processor affinities at startup.
- Disk module conditioning scripts that create and configure RAID
arrays, label and partition virtual drives, create EXT4 file systems,
and optimize file system parameters.
Release 0.4.1 9/20/2011
-----------------------
Committer: David Lapsley<[email protected]>
Added control plane and client to the build. Extended build system
to do complete build and install.
- Incorporated mark6 control plane code (inherited from previous
dimino6 work).
- Incorporated tstmark6 control plane client.
- Extended build system to do full build and install in
/opt/haystack/mark6.
Release 0.4.2 9/28/2011
-----------------------
Committer: David Lapsley<[email protected]>
Validated 16 Gbps data capture functionality. Recording full
ethernet frames for robust on-disk framing.
- Full ethernet frame capture for robust on-disk framing and ease of
manipulation. Data capture files stored in standard "pcap" format
to facilitate manipulation with tcpdump, editcap, and other pcap
based applications.
- Full application installation (binaries, scripts, configuration,
and logs) into /opt/mit/mark6 directory.
- Addition of disk2vlbi tool for converting captured pcap file format
to native VLBI format (i.e. vdif, m5b).
- Migration of BufferPool functionality into FileWriter class to
simplify/robustify code.
- Removal of BufferPool class.
- Addition of blocking semantics to disk buffer allocation (through
FileWriter::buffer_malloc() method.
- Enhanced instrumentation (buffer level and packet drop metrics added).
- Automatic ethernet IRQ sensing in net2disk-run script.
- Incorporation of disk tools into distributed application (eventually will
be integrated into control plane as conditioning/test scripts).
Release 0.5.0 10/24/2011
------------------------
Committer: David Lapsley<[email protected]>
- Version to be used for 16 Gbps demonstration.
Release 0.6.0 11/13/2011
------------------------
Committer: David Lapsley<[email protected]>
- First public release!
- Includes 16 Gbps-capable data plane demonstrated on 10/24/2011 (and
extensively tested).
- Incorporates Control Plane skeleton into build.
- Includes substantial online documentation (http://www.vdas.org).
- Released under GPLv2.