Project

General

Profile

Statistics
| Branch: | Tag: | Revision:

gdp / README-compiling.md @ master

History | View | Annotate | Download (11.4 KB)

1 7f982ac3 Eric Allman
% Compiling The Global Dataplane Software
2 b2714365 Eric Allman
3 390904c6 Eric Allman
**NOTE:** The page at
4
<https://gdp.cs.berkeley.edu/redmine/projects/gdp/wiki/Compiling_the_GDP_from_Source>
5
is likely to be more up-to-date than this file.
6
7 b2714365 Eric Allman
This directory contains the source code for the Global Data Plane
8
(GDP) with several language bindings.  The primary binding is for
9
C (and will work with C++).  The others include Python, Java, and
10 632861ce Eric Allman
Javascript.  *[[As of 2018-12-07 the GDPv2 Javascript bindings are
11
not known to work.]]*
12 b2714365 Eric Allman
13
NOTE: these instructions assume you are starting with the source
14
distribution.  This is not appropriate if you are installing the
15
Debian package.  Since you are reading this, you presumably already
16
have the source code.  If not, you can get the GDP source distribution
17
using one of the following commands:
18
19 91040310 Eric Allman
	git clone git://repo.eecs.berkeley.edu/projects/swarmlab/gdp.git
20 74875cb2 Eric Allman
	git clone https://repo.eecs.berkeley.edu/git/projects/swarmlab/gdp.git
21
	git clone repoman@repo.eecs.berkeley.edu:projects/swarmlab/gdp.git
22 b2714365 Eric Allman
23 91040310 Eric Allman
The first form gives you public, read-only access, while the other
24
two require that you have an account on the EECS repository.
25
The third is only available if you have registered your public
26 2d3b765a Eric Allman
ssh key with `repo.eecs.berkeley.edu`.
27 b2714365 Eric Allman
28 632861ce Eric Allman
**NOTE WELL:** if you are compiling from the GDP source tree, you
29
*should* request an account on <https://gdp.cs.berkeley.edu> and
30
subscribe to the news feed for the GDP project.  This is the only
31
way to be informed when new versions are released.
32
33 b2714365 Eric Allman
At the moment, compiles work on many platforms, including Debian,
34
RedHat, MacOS, and FreeBSD.  However, some other GDP-related packages
35
work on Debian only (which includes Ubuntu), so you may have
36 fa7ea866 Eric Allman
difficulties outside the main source tree.  See the section on
37 632861ce Eric Allman
Operating System Quirks below for some hints.  Some subsystems only
38
work on Debian-based systems.
39 b2714365 Eric Allman
40
Installing Requisite Packages
41 7f982ac3 Eric Allman
=============================
42 b2714365 Eric Allman
43
When compiling from source code, there is no distinction between
44
client and server packages; both are compiled every time.  For this
45
reason, you must install all requisite packages before compiling.
46
The easiest way to do this is to run the `adm/gdp-setup.sh`
47
script.
48
49
Note that on some systems you may need to install the compile suite
50 fa7ea866 Eric Allman
as well.
51 b2714365 Eric Allman
52
Compilation
53 7f982ac3 Eric Allman
===========
54 b2714365 Eric Allman
55
Compiling the primary code tree should just be a matter of typing
56 632861ce Eric Allman
`make` in the root of the `gdp` tree (this assumes that you have already
57
done `sh adm/gdp-setup.sh` to install the requisite packages).  If you
58
want to clear out old cruft, use `make clean all`.  You can install
59
the packages into the system tree (by default, `/usr/bin`, `/usr/sbin`,
60
and `/usr/lib`) using `make install`.  If you prefer installing into
61 b2714365 Eric Allman
something other than the main tree, set the `LOCALROOT` variable
62
on the `make` command line.  For example:
63
64 74875cb2 Eric Allman
	make clean install LOCALROOT=/usr/local
65 b2714365 Eric Allman
66
It is not necessary to install the code for testing and debugging.
67
68
If you are going to be debugging it can be convenient to use
69
`O=` on the `make` command line.  This will turn off optimization,
70 632861ce Eric Allman
which makes debugger output more understandable.
71 b2714365 Eric Allman
72
Note: gcc on linux has a bug that causes it to complain about
73
non-constant expressions in an initializer when the `-std=c99`
74
flag is given.  Those same expressions are constant in Clang
75
and even in gcc without the `-std=c99` flag.  As a result of
76 15153231 Eric Allman
this problem, we do not use the `-std=c99` flag by default, but
77 b2714365 Eric Allman
this means that not all features of C99 are available.
78
If you want full C99, use `STD=-std=c99` on the make command
79
line.
80
81
Further note: At least some versions of gcc give warnings
82
about ignored return values even when the function call has
83 632861ce Eric Allman
been explicitly `void`ed.  We know about this and do not
84 b2714365 Eric Allman
consider it to be a bug in the GDP code.  If these warnings
85
bother you we recommend installing clang and using that
86
compiler.  (Hint: it gives much better error messages and
87
catches things that gcc does not.)
88
89 3a7c9df9 Eric Allman
### Compilation Flags
90
91
There are a few compilation flags you can use to turn on additional
92
debugging.  You can generally do this using `O=-D`flag on the
93
`make` command line; this also turns off optimization so that
94
debuggers can give better information.
95
96 632861ce Eric Allman
* *EP_OPT_EXTENDED_MUTEX_CHECK*: Check for cases such as locking
97 3a7c9df9 Eric Allman
  mutexes that are already locked.  Also enables some assertions
98
  that check lock status.  Only works on systems using the NTPL
99
  implementation (which means most versions of Linux).  This breaks
100
  information hiding and hence may fail if the NPTL implementation
101
  changes.
102 632861ce Eric Allman
* *GDP_OPT_EXTENDED_CACHE_CHECK*: This does some additional
103 3a7c9df9 Eric Allman
  checking on the GCL cache.  Notably, it fails if a loop in one of
104
  the linked lists is detected.  This can be expensive, so use it
105
  sparingly.
106
107 b2714365 Eric Allman
### Other language bindings
108
109
In addition to C, there is support for Python, Java, and
110
Javascript.  These bindings are all in the `lang` subtree.
111
See the instructions in those directories for compiling.
112
113 fa7ea866 Eric Allman
Operating System Quirks
114 7f982ac3 Eric Allman
=======================
115 fa7ea866 Eric Allman
116
### MacOS
117
118 15153231 Eric Allman
If you are trying to compile on MacOS you will need to install
119 fa7ea866 Eric Allman
Xcode from the App Store to get the compilers, libraries, and
120 15153231 Eric Allman
build tools you will need.  Instructions for doing this are
121 632861ce Eric Allman
available at <https://www.macports.org/install.php>.
122 fa7ea866 Eric Allman
123
Other packages are installed by `adm/gdp-setup.sh`.  Note that
124
this script will try to determine if you are using `brew` or
125 15153231 Eric Allman
`macports`.  Of the two, `macports` is better understood and
126
supports more required packages.  If you use `homebrew` you will
127
have to install some packages by hand.
128 29e6f6d0 Christopher Brooks
129 632861ce Eric Allman
To install `macports`, see <https://www.macports.org/install.php>.
130 15153231 Eric Allman
It comes as standard binary packages for most versions: download
131
the `.dmg` file, double click on the package, and proceed.
132 29e6f6d0 Christopher Brooks
133 632861ce Eric Allman
To install brew, see <http://brew.sh/> and run
134 15153231 Eric Allman
   /usr/bin/ruby -e \"$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)\"
135 29e6f6d0 Christopher Brooks
136 15153231 Eric Allman
Unfortunately, neither of them has all the modules you may need
137
if you are compiling everything, so you may have to download other
138
packages from source code.  For example, `macports` does not
139
support `mosquitto`, which is necessary if you are compiling the
140
MQTT-GDP gateway code (which is not part of the default build).
141
Conversely, `brew` does not support Avahi, which _is_ part of
142
the default build.
143 29e6f6d0 Christopher Brooks
144 15153231 Eric Allman
If you cannot use `macports` but you want Zeroconf, you can install
145
Avahi by hand using:
146 29e6f6d0 Christopher Brooks
147
    wget https://github.com/lathiat/avahi/releases/download/v0.6.32/avahi-0.6.32.tar.gz
148
    tar -zxf avahi-0.6.32.tar.gz
149
    cd avahi-0.6.32
150 15153231 Eric Allman
   ./configure --disable-qt4 --disable-qt3 --disable-gtk \
151
        --disable-gtk3 --disable-gdbm --disable-pygtk \
152
		--disable-python-dbus --disable-mono 
153 29e6f6d0 Christopher Brooks
    make -k
154 f98854fe Eric Allman
155 15153231 Eric Allman
Then `cd` to the `avahi-0.6.32` directory and run:
156
157
    sudo make -k install
158
159
Note also that `0.6.32` is the current version as of this writing;
160
you may need to find the latest version if that one becomes obsolete.
161
162
As an alternative, you can remove Zeroconf from the compilation
163
entirely using:
164
165 6ef85f26 Christopher Brooks
    make all_noavahi
166 f98854fe Eric Allman
167 15153231 Eric Allman
Be aware that this will increase the need for manual configuration
168
at all user sites.
169 7acb8582 Eric Allman
170 5f23913e Christopher Brooks
If make fails because openssl/evp.h is not found, then find evp.h
171
with:
172
173
   find /usr/local/Cellar -name "evp.h"
174
175
For example:
176
177
   bash-3.2$ find /usr/local/Cellar -name "evp.h"
178
   /usr/local/Cellar/openssl/1.0.2k/include/openssl/evp.h
179
180
Use the directory above the include directory for the value
181
of LOCAL2:
182
183
   make LOCAL2=/usr/local/Cellar/openssl/1.0.2k all_noavahi
184
185 fa7ea866 Eric Allman
### Red Hat
186
187 632861ce Eric Allman
Debian is the preferred Linux distribution for the GDP.  We used
188
to have a nightly RedHat build, but that group shut down, so that
189
platform should be considered unsupported.
190 fa7ea866 Eric Allman
191
### FreeBSD
192
193
We do attempt to compile on FreeBSD occasionally in an attempt
194
to promote portability, but some of the other optional packages
195
do not compile or run on FreeBSD.  However, the base code
196 632861ce Eric Allman
should compile.  Please let us know if you have problems; however,
197
it is not a high priority.
198 fa7ea866 Eric Allman
199 b2714365 Eric Allman
Next Steps
200 7f982ac3 Eric Allman
==========
201 b2714365 Eric Allman
202
If you just want to use the GDP client programs, continue
203
reading README.md
204
205
If you intend to install and maintain your own GDP routers
206
and/or log servers, please continue with README-admin.md.
207
208
If you plan on debugging the GDP code itself, continue with
209
README-developers.md.
210
211
Directory Structure
212 7f982ac3 Eric Allman
===================
213 b2714365 Eric Allman
214
The following is a brief explanation of the subdirectories
215
contained in this source tree.
216
217 632861ce Eric Allman
* `ep`: A library of C utility functions.  This is a stripped
218 b2714365 Eric Allman
	down version of a library I wrote several years ago.
219 15153231 Eric Allman
	If you look at the code you will see vestiges of some
220 b2714365 Eric Allman
	of the stripped out functions.  I plan on cleaning
221
	this version up and releasing it again.
222
223 632861ce Eric Allman
* `gdp`: A library for GDP manipulation.  This is the library
224 b2714365 Eric Allman
	that applications must link to access the GDP.
225
226 632861ce Eric Allman
* `gdplogd`: The GDP log daemon.  This implements physical
227 b2714365 Eric Allman
	(on disk) logs for the GDP.  The implementation is
228
	still fairly simplistic.  It depends on a routing
229
	layer (currently gdp_router, in a separate repository).
230
231 632861ce Eric Allman
* `services`: various system services.  Most of these are essential
232
  if you want a local cluster.  If you are tying in to the Berkeley
233
  cluster you can just use what we run.  At the moment, services
234
  are mostly cluster-specific, so setting up your own cluster will
235
  probably require hand modification of the configurations.
236
237 79c3499d Rick Pratt
* `services/gdp-ribd`: The GDP routing information base daemon.
238
  Accepts router network adjacency updates (derived from GDP
239
  advertisements and withdrawals), maintains a network graph, and
240
  responds to router next hop queries. All adjacencies are pairs of
241
  GDPnames (256-bit binary numbers).  The gdp-ribd is an interim
242
  solution which facilitated development of the new GDP v2 new router,
243
  thus one should expect the implementation will be replaced in part
244
  or whole in the future.
245 632861ce Eric Allman
246
* `services/log-creation`: log creation assistant.  Accepts a log
247
  creation request generated by an application and selects a
248
  destination log server to host the log.  Also updates the
249
  Human-Oriented Name to GDPname Directory (HONGD) based on log
250
  metadata.  Curently fairly simplistic, and must be configured
251
  for each GDP cluster.
252
253
* `apps`: application programs.  At the moment these include both
254
  user-oriented applications (which normally live in `/usr/bin`) and
255
  administrator-specific programs (which normally live in `/usr/sbin`).
256
  These should be better segregated.
257
258
* `test`: test programs.  At the moment somewhat random.  It should
259
  include more comprehensive suites.
260
261
* `doc`: Some documentation, woefully incomplete.  None the less,
262
  please look here first.  We are constantly striving to improve it.
263
  Also see <https://gdp.cs.berkeley.edu> for more up-to-date detail.
264
265
* `examples`: Some example programs, intended be usable as
266
  tutorials.  These are mostly not up to date, but may still be
267
  useful.
268
269
* `lang`: sub-directories with language-specific application
270 b2714365 Eric Allman
	programs and supporting code.
271
272 632861ce Eric Allman
* `lang/java`: Java-specific apps and libraries.
273 b2714365 Eric Allman
274 632861ce Eric Allman
* `lang/js`: JavaScript-specific apps and libraries.  Also contains
275 79c3499d Rick Pratt
	the Node.js/JS GDP RESTful interface code.  Not to be confused
276
	with the 'apps/gdp-rest.c` GDP RESTful server. See associated
277 b2714365 Eric Allman
	README files for details.
278
279 632861ce Eric Allman
* `lang/python`: Python-specific apps and libraries.  See the
280 b2714365 Eric Allman
	associated README file for details.
281
282 632861ce Eric Allman
* `scgilib`: An updated version of the SCGI code from
283
	[`http://www.xamuel.com/scgilib/`](http://www.xameul.com/scgilib/).
284
	SCGI permits a web server to access outside programs by opening
285
	a socket in a manner much more efficient than basic
286 79c3499d Rick Pratt
	CGI fork/exec.  This is only used for the 'apps/gdp-rest.c`
287
	GDP RESTful server.
288 632861ce Eric Allman
289 b2714365 Eric Allman
<!-- vim: set ai sw=4 sts=4 ts=4 : -->
290 7f982ac3 Eric Allman
<!-- Use "pandoc -s -o README-compiling.html README-compiling.md"
291 632861ce Eric Allman
to process this to HTML -->