Project

General

Profile

Statistics
| Branch: | Tag: | Revision:

gdp / README-compiling.md @ master

History | View | Annotate | Download (11.5 KB)

1 b2714365 Eric Allman
2
COMPILING THE GLOBAL DATAPLANE SOFTWARE
3
=======================================
4
5 390904c6 Eric Allman
**NOTE:** The page at
6
<https://gdp.cs.berkeley.edu/redmine/projects/gdp/wiki/Compiling_the_GDP_from_Source>
7
is likely to be more up-to-date than this file.
8
9 b2714365 Eric Allman
This directory contains the source code for the Global Data Plane
10
(GDP) with several language bindings.  The primary binding is for
11
C (and will work with C++).  The others include Python, Java, and
12 632861ce Eric Allman
Javascript.  *[[As of 2018-12-07 the GDPv2 Javascript bindings are
13
not known to work.]]*
14 b2714365 Eric Allman
15
NOTE: these instructions assume you are starting with the source
16
distribution.  This is not appropriate if you are installing the
17
Debian package.  Since you are reading this, you presumably already
18
have the source code.  If not, you can get the GDP source distribution
19
using one of the following commands:
20
21 91040310 Eric Allman
	git clone git://repo.eecs.berkeley.edu/projects/swarmlab/gdp.git
22 74875cb2 Eric Allman
	git clone https://repo.eecs.berkeley.edu/git/projects/swarmlab/gdp.git
23
	git clone repoman@repo.eecs.berkeley.edu:projects/swarmlab/gdp.git
24 b2714365 Eric Allman
25 91040310 Eric Allman
The first form gives you public, read-only access, while the other
26
two require that you have an account on the EECS repository.
27
The third is only available if you have registered your public
28 2d3b765a Eric Allman
ssh key with `repo.eecs.berkeley.edu`.
29 b2714365 Eric Allman
30 632861ce Eric Allman
**NOTE WELL:** if you are compiling from the GDP source tree, you
31
*should* request an account on <https://gdp.cs.berkeley.edu> and
32
subscribe to the news feed for the GDP project.  This is the only
33
way to be informed when new versions are released.
34
35 b2714365 Eric Allman
At the moment, compiles work on many platforms, including Debian,
36
RedHat, MacOS, and FreeBSD.  However, some other GDP-related packages
37
work on Debian only (which includes Ubuntu), so you may have
38 fa7ea866 Eric Allman
difficulties outside the main source tree.  See the section on
39 632861ce Eric Allman
Operating System Quirks below for some hints.  Some subsystems only
40
work on Debian-based systems.
41 b2714365 Eric Allman
42
Installing Requisite Packages
43
-----------------------------
44
45
When compiling from source code, there is no distinction between
46
client and server packages; both are compiled every time.  For this
47
reason, you must install all requisite packages before compiling.
48
The easiest way to do this is to run the `adm/gdp-setup.sh`
49
script.
50
51
Note that on some systems you may need to install the compile suite
52 fa7ea866 Eric Allman
as well.
53 b2714365 Eric Allman
54
Compilation
55
-----------
56
57
Compiling the primary code tree should just be a matter of typing
58 632861ce Eric Allman
`make` in the root of the `gdp` tree (this assumes that you have already
59
done `sh adm/gdp-setup.sh` to install the requisite packages).  If you
60
want to clear out old cruft, use `make clean all`.  You can install
61
the packages into the system tree (by default, `/usr/bin`, `/usr/sbin`,
62
and `/usr/lib`) using `make install`.  If you prefer installing into
63 b2714365 Eric Allman
something other than the main tree, set the `LOCALROOT` variable
64
on the `make` command line.  For example:
65
66 74875cb2 Eric Allman
	make clean install LOCALROOT=/usr/local
67 b2714365 Eric Allman
68
It is not necessary to install the code for testing and debugging.
69
70
If you are going to be debugging it can be convenient to use
71
`O=` on the `make` command line.  This will turn off optimization,
72 632861ce Eric Allman
which makes debugger output more understandable.
73 b2714365 Eric Allman
74
Note: gcc on linux has a bug that causes it to complain about
75
non-constant expressions in an initializer when the `-std=c99`
76
flag is given.  Those same expressions are constant in Clang
77
and even in gcc without the `-std=c99` flag.  As a result of
78 15153231 Eric Allman
this problem, we do not use the `-std=c99` flag by default, but
79 b2714365 Eric Allman
this means that not all features of C99 are available.
80
If you want full C99, use `STD=-std=c99` on the make command
81
line.
82
83
Further note: At least some versions of gcc give warnings
84
about ignored return values even when the function call has
85 632861ce Eric Allman
been explicitly `void`ed.  We know about this and do not
86 b2714365 Eric Allman
consider it to be a bug in the GDP code.  If these warnings
87
bother you we recommend installing clang and using that
88
compiler.  (Hint: it gives much better error messages and
89
catches things that gcc does not.)
90
91 3a7c9df9 Eric Allman
### Compilation Flags
92
93
There are a few compilation flags you can use to turn on additional
94
debugging.  You can generally do this using `O=-D`flag on the
95
`make` command line; this also turns off optimization so that
96
debuggers can give better information.
97
98 632861ce Eric Allman
* *EP_OPT_EXTENDED_MUTEX_CHECK*: Check for cases such as locking
99 3a7c9df9 Eric Allman
  mutexes that are already locked.  Also enables some assertions
100
  that check lock status.  Only works on systems using the NTPL
101
  implementation (which means most versions of Linux).  This breaks
102
  information hiding and hence may fail if the NPTL implementation
103
  changes.
104 632861ce Eric Allman
* *GDP_OPT_EXTENDED_CACHE_CHECK*: This does some additional
105 3a7c9df9 Eric Allman
  checking on the GCL cache.  Notably, it fails if a loop in one of
106
  the linked lists is detected.  This can be expensive, so use it
107
  sparingly.
108
109 b2714365 Eric Allman
### Other language bindings
110
111
In addition to C, there is support for Python, Java, and
112
Javascript.  These bindings are all in the `lang` subtree.
113
See the instructions in those directories for compiling.
114
115 fa7ea866 Eric Allman
Operating System Quirks
116
-----------------------
117
118
### MacOS
119
120 15153231 Eric Allman
If you are trying to compile on MacOS you will need to install
121 fa7ea866 Eric Allman
Xcode from the App Store to get the compilers, libraries, and
122 15153231 Eric Allman
build tools you will need.  Instructions for doing this are
123 632861ce Eric Allman
available at <https://www.macports.org/install.php>.
124 fa7ea866 Eric Allman
125
Other packages are installed by `adm/gdp-setup.sh`.  Note that
126
this script will try to determine if you are using `brew` or
127 15153231 Eric Allman
`macports`.  Of the two, `macports` is better understood and
128
supports more required packages.  If you use `homebrew` you will
129
have to install some packages by hand.
130 29e6f6d0 Christopher Brooks
131 632861ce Eric Allman
To install `macports`, see <https://www.macports.org/install.php>.
132 15153231 Eric Allman
It comes as standard binary packages for most versions: download
133
the `.dmg` file, double click on the package, and proceed.
134 29e6f6d0 Christopher Brooks
135 632861ce Eric Allman
To install brew, see <http://brew.sh/> and run
136 15153231 Eric Allman
   /usr/bin/ruby -e \"$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)\"
137 29e6f6d0 Christopher Brooks
138 15153231 Eric Allman
Unfortunately, neither of them has all the modules you may need
139
if you are compiling everything, so you may have to download other
140
packages from source code.  For example, `macports` does not
141
support `mosquitto`, which is necessary if you are compiling the
142
MQTT-GDP gateway code (which is not part of the default build).
143
Conversely, `brew` does not support Avahi, which _is_ part of
144
the default build.
145 29e6f6d0 Christopher Brooks
146 15153231 Eric Allman
If you cannot use `macports` but you want Zeroconf, you can install
147
Avahi by hand using:
148 29e6f6d0 Christopher Brooks
149
    wget https://github.com/lathiat/avahi/releases/download/v0.6.32/avahi-0.6.32.tar.gz
150
    tar -zxf avahi-0.6.32.tar.gz
151
    cd avahi-0.6.32
152 15153231 Eric Allman
   ./configure --disable-qt4 --disable-qt3 --disable-gtk \
153
        --disable-gtk3 --disable-gdbm --disable-pygtk \
154
		--disable-python-dbus --disable-mono 
155 29e6f6d0 Christopher Brooks
    make -k
156 f98854fe Eric Allman
157 15153231 Eric Allman
Then `cd` to the `avahi-0.6.32` directory and run:
158
159
    sudo make -k install
160
161
Note also that `0.6.32` is the current version as of this writing;
162
you may need to find the latest version if that one becomes obsolete.
163
164
As an alternative, you can remove Zeroconf from the compilation
165
entirely using:
166
167 6ef85f26 Christopher Brooks
    make all_noavahi
168 f98854fe Eric Allman
169 15153231 Eric Allman
Be aware that this will increase the need for manual configuration
170
at all user sites.
171 7acb8582 Eric Allman
172 5f23913e Christopher Brooks
If make fails because openssl/evp.h is not found, then find evp.h
173
with:
174
175
   find /usr/local/Cellar -name "evp.h"
176
177
For example:
178
179
   bash-3.2$ find /usr/local/Cellar -name "evp.h"
180
   /usr/local/Cellar/openssl/1.0.2k/include/openssl/evp.h
181
182
Use the directory above the include directory for the value
183
of LOCAL2:
184
185
   make LOCAL2=/usr/local/Cellar/openssl/1.0.2k all_noavahi
186
187 fa7ea866 Eric Allman
### Red Hat
188
189 632861ce Eric Allman
Debian is the preferred Linux distribution for the GDP.  We used
190
to have a nightly RedHat build, but that group shut down, so that
191
platform should be considered unsupported.
192 fa7ea866 Eric Allman
193
### FreeBSD
194
195
We do attempt to compile on FreeBSD occasionally in an attempt
196
to promote portability, but some of the other optional packages
197
do not compile or run on FreeBSD.  However, the base code
198 632861ce Eric Allman
should compile.  Please let us know if you have problems; however,
199
it is not a high priority.
200 fa7ea866 Eric Allman
201 b2714365 Eric Allman
Next Steps
202
----------
203
204
If you just want to use the GDP client programs, continue
205
reading README.md
206
207
If you intend to install and maintain your own GDP routers
208
and/or log servers, please continue with README-admin.md.
209
210
If you plan on debugging the GDP code itself, continue with
211
README-developers.md.
212
213
Directory Structure
214
-------------------
215
216
The following is a brief explanation of the subdirectories
217
contained in this source tree.
218
219 632861ce Eric Allman
* `ep`: A library of C utility functions.  This is a stripped
220 b2714365 Eric Allman
	down version of a library I wrote several years ago.
221 15153231 Eric Allman
	If you look at the code you will see vestiges of some
222 b2714365 Eric Allman
	of the stripped out functions.  I plan on cleaning
223
	this version up and releasing it again.
224
225 632861ce Eric Allman
* `gdp`: A library for GDP manipulation.  This is the library
226 b2714365 Eric Allman
	that applications must link to access the GDP.
227
228 632861ce Eric Allman
* `gdplogd`: The GDP log daemon.  This implements physical
229 b2714365 Eric Allman
	(on disk) logs for the GDP.  The implementation is
230
	still fairly simplistic.  It depends on a routing
231
	layer (currently gdp_router, in a separate repository).
232
233 632861ce Eric Allman
* `services`: various system services.  Most of these are essential
234
  if you want a local cluster.  If you are tying in to the Berkeley
235
  cluster you can just use what we run.  At the moment, services
236
  are mostly cluster-specific, so setting up your own cluster will
237
  probably require hand modification of the configurations.
238
239 79c3499d Rick Pratt
* `services/gdp-ribd`: The GDP routing information base daemon.
240
  Accepts router network adjacency updates (derived from GDP
241
  advertisements and withdrawals), maintains a network graph, and
242
  responds to router next hop queries. All adjacencies are pairs of
243
  GDPnames (256-bit binary numbers).  The gdp-ribd is an interim
244
  solution which facilitated development of the new GDP v2 new router,
245
  thus one should expect the implementation will be replaced in part
246
  or whole in the future.
247 632861ce Eric Allman
248
* `services/log-creation`: log creation assistant.  Accepts a log
249
  creation request generated by an application and selects a
250
  destination log server to host the log.  Also updates the
251
  Human-Oriented Name to GDPname Directory (HONGD) based on log
252
  metadata.  Curently fairly simplistic, and must be configured
253
  for each GDP cluster.
254
255
* `apps`: application programs.  At the moment these include both
256
  user-oriented applications (which normally live in `/usr/bin`) and
257
  administrator-specific programs (which normally live in `/usr/sbin`).
258
  These should be better segregated.
259
260
* `test`: test programs.  At the moment somewhat random.  It should
261
  include more comprehensive suites.
262
263
* `doc`: Some documentation, woefully incomplete.  None the less,
264
  please look here first.  We are constantly striving to improve it.
265
  Also see <https://gdp.cs.berkeley.edu> for more up-to-date detail.
266
267
* `examples`: Some example programs, intended be usable as
268
  tutorials.  These are mostly not up to date, but may still be
269
  useful.
270
271
* `lang`: sub-directories with language-specific application
272 b2714365 Eric Allman
	programs and supporting code.
273
274 632861ce Eric Allman
* `lang/java`: Java-specific apps and libraries.
275 b2714365 Eric Allman
276 632861ce Eric Allman
* `lang/js`: JavaScript-specific apps and libraries.  Also contains
277 79c3499d Rick Pratt
	the Node.js/JS GDP RESTful interface code.  Not to be confused
278
	with the 'apps/gdp-rest.c` GDP RESTful server. See associated
279 b2714365 Eric Allman
	README files for details.
280
281 632861ce Eric Allman
* `lang/python`: Python-specific apps and libraries.  See the
282 b2714365 Eric Allman
	associated README file for details.
283
284 632861ce Eric Allman
* `scgilib`: An updated version of the SCGI code from
285
	[`http://www.xamuel.com/scgilib/`](http://www.xameul.com/scgilib/).
286
	SCGI permits a web server to access outside programs by opening
287
	a socket in a manner much more efficient than basic
288 79c3499d Rick Pratt
	CGI fork/exec.  This is only used for the 'apps/gdp-rest.c`
289
	GDP RESTful server.
290 632861ce Eric Allman
291 b2714365 Eric Allman
<!-- vim: set ai sw=4 sts=4 ts=4 : -->
292 390904c6 Eric Allman
<!-- Use "pandoc -sS -o README-compiling.html README-compiling.md"
293 632861ce Eric Allman
to process this to HTML -->