-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #41 from DICL/update-1.8.0
Updated VDFS to VeloxDFS 1.8.0 There are several changes: -RemoteMR/PeerMR is substituted by TaskExecutor and TaskExecutorRouter. -Mapreduce routines only depends on the DFS client API. -VeloxDFS and VeloxMR nodes runs in two different process. -Reducer write its output to its local machine (Vague optimization). -Removed lots of deprecated source code It is not solid stable but the most stable we have gotten so far
- Loading branch information
Showing
173 changed files
with
6,135 additions
and
5,981 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,57 @@ | ||
| --- | ||
| Language: Cpp | ||
| # BasedOnStyle: Google | ||
| AccessModifierOffset: -1 | ||
| ConstructorInitializerIndentWidth: 4 | ||
| AlignEscapedNewlinesLeft: true | ||
| AlignTrailingComments: true | ||
| AllowAllParametersOfDeclarationOnNextLine: true | ||
| AllowShortBlocksOnASingleLine: false | ||
| AllowShortIfStatementsOnASingleLine: true | ||
| AllowShortLoopsOnASingleLine: true | ||
| AllowShortFunctionsOnASingleLine: All | ||
| AlwaysBreakTemplateDeclarations: true | ||
| AlwaysBreakBeforeMultilineStrings: true | ||
| BreakBeforeBinaryOperators: false | ||
| BreakBeforeTernaryOperators: true | ||
| BreakConstructorInitializersBeforeComma: false | ||
| BinPackParameters: true | ||
| ColumnLimit: 80 | ||
| ConstructorInitializerAllOnOneLineOrOnePerLine: true | ||
| DerivePointerAlignment: true | ||
| ExperimentalAutoDetectBinPacking: false | ||
| IndentCaseLabels: true | ||
| IndentWrappedFunctionNames: false | ||
| IndentFunctionDeclarationAfterType: false | ||
| MaxEmptyLinesToKeep: 1 | ||
| KeepEmptyLinesAtTheStartOfBlocks: false | ||
| NamespaceIndentation: None | ||
| ObjCSpaceAfterProperty: false | ||
| ObjCSpaceBeforeProtocolList: false | ||
| PenaltyBreakBeforeFirstCallParameter: 1 | ||
| PenaltyBreakComment: 300 | ||
| PenaltyBreakString: 1000 | ||
| PenaltyBreakFirstLessLess: 120 | ||
| PenaltyExcessCharacter: 1000000 | ||
| PenaltyReturnTypeOnItsOwnLine: 200 | ||
| PointerAlignment: Left | ||
| SpacesBeforeTrailingComments: 2 | ||
| Cpp11BracedListStyle: true | ||
| Standard: Auto | ||
| IndentWidth: 2 | ||
| TabWidth: 8 | ||
| UseTab: Never | ||
| BreakBeforeBraces: Attach | ||
| SpacesInParentheses: false | ||
| SpacesInAngles: false | ||
| SpaceInEmptyParentheses: false | ||
| SpacesInCStyleCastParentheses: false | ||
| SpacesInContainerLiterals: true | ||
| SpaceBeforeAssignmentOperators: true | ||
| ContinuationIndentWidth: 4 | ||
| CommentPragmas: '^ IWYU pragma:' | ||
| ForEachMacros: [ foreach, Q_FOREACH, BOOST_FOREACH ] | ||
| SpaceBeforeParens: ControlStatements | ||
| DisableFormat: false | ||
| ... | ||
|
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,7 +1,18 @@ | ||
| This PR Fixes DICL/EclipseMETA#(Which issue?) | ||
|
|
||
| Checklist | ||
| - [ ] Follows Google Coding Style. | ||
| - [ ] At least one reviewer approved (for non-trivial changes). | ||
| ## BRIEF | ||
|
|
||
| ## STATUS | ||
| - [ ] Its implemented. | ||
| - [ ] It compiles. | ||
| - [ ] Its tested. | ||
| - [ ] Its refactored. | ||
|
|
||
| --- | ||
| Make sure that you squeeze all your commits before merging to master. | ||
| You might want to use the following command: | ||
|
|
||
| $ git rebase -i #hash key of base commit | ||
|
|
||
| --- | ||
| ## EXTRA |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,8 +1,9 @@ | ||
| [](https://travis-ci.org/DICL/VeloxMR) | ||
| [](https://dicl.slack.com/messages/general/) | ||
| <a href="https://zenhub.com"><img src="https://raw.githubusercontent.com/ZenHubIO/support/master/zenhub-badge.png"></a> | ||
| [](https://zenhub.com) | ||
| [](https://github.com/DICL/VeloxMR) | ||
| BRIEFING | ||
| --- | ||
| VeloxMR {#mainpage} | ||
| ======== | ||
|
|
||
| VeloxMR is a MapReduce framework implemented in C++ on the top of VeloxDF | ||
|
|
@@ -16,17 +17,19 @@ USAGE | |
| ===== | ||
| VeloxMR default launcher is not included in this repository, if you want to use it you can find it [here][eclipsed]. | ||
|
|
||
| The reason to not to include the launcher inside the package is to let the user to chose any launcher, options are: | ||
| The reason not to include the launcher inside the package is to let the user to choose any launcher or service managers such as: | ||
| - systemd/init.d | ||
| - puppet/chef/salt | ||
|
|
||
| Once the system is running, you can interact with EclipseDFS with the following commands: | ||
| Once the system is up and running, you can interact with VeloxDFS with the following commands: | ||
| ``` | ||
| $ dfs ls|put|get|rm|format|show | ||
| $ dfs put|get|cat|ls|rm|format|pget|update|append | ||
| ``` | ||
|
|
||
| COMPILING & INSTALLING | ||
| ===================== | ||
| ====================== | ||
|
|
||
| _Detailed information can be found in the wiki of this repository_ | ||
|
|
||
| Compiling requirements | ||
| ---------------------- | ||
|
|
@@ -41,11 +44,11 @@ For single user installation for developers | |
|
|
||
| $ mkdir -p local_eclipse/{tmp,sandbox} # Create a sandbox directories | ||
| $ cd local_eclipse # enter in the directory | ||
| $ git clone [email protected]:DICL/EclipseDFS.git # Clone the project from github | ||
| $ cd EclipseDFS | ||
| $ git clone [email protected]:DICL/VeloxDFS.git # Clone the project from github | ||
| $ cd VeloxDFS | ||
| $ sh autogen.sh # Generate configure script | ||
| $ cd ../tmp # Go to building folder | ||
| $ sh ../EclipseDFS/configure --prefix=`pwd`/../sandbox # Check requirements and generate the Makefile | ||
| $ sh ../VeloxDFS/configure --prefix=`pwd`/../sandbox # Check requirements and generate the Makefile | ||
|
|
||
| # If you get a boost error go the FAQ section of the README | ||
|
|
||
|
|
@@ -57,11 +60,28 @@ Now edit in your **~/.bashrc** or **~/.profile**: | |
| export PATH="/home/*..PATH/To/eclipse/..*/sandbox/bin":$PATH | ||
| export LIBRARY_PATH="/home/*..PATH/To/eclipse/..*/sandbox/lib" | ||
| export C_INCLUDE_PATH="/home/*..PATH/To/eclipse/..*/sandbox/include" | ||
| export MANPATH=`manpath`:/home*..PATH/To/eclipse/..*/sandbox/share/man | ||
|
|
||
| For the configuration refer to the manpage: | ||
|
|
||
| $ man eclipsefs | ||
| Default settings for VELOXDFS | ||
| ----------------------------- | ||
|
|
||
| "log" : { | ||
| "type" : "LOG_LOCAL6" | ||
| "name" : "ECLIPSE" | ||
| "mask" : "DEBUG" | ||
| }, | ||
|
|
||
| "cache" : { | ||
| "numbin" : 100, | ||
| "size" : 200000, | ||
| "concurrency" : 1 | ||
| }, | ||
|
|
||
| "filesystem" : { | ||
| "block" : 137438953, | ||
| "buffer" : 512, | ||
| "replica" : 1 | ||
| } | ||
|
|
||
| FAQ | ||
| --- | ||
|
|
@@ -70,27 +90,18 @@ FAQ | |
| - _Answer_ : It probably means that you do not have boost library installed in | ||
| the default location, in such case you should specify the boost library location. | ||
| ``` | ||
| sh ../EclipseDFS/configure --prefix ~/sandbox --with-boost=/usr/local --with-boost-libdir=/usr/local/lib | ||
| sh ../VeloxDFS/configure --prefix ~/sandbox --with-boost=/usr/local --with-boost-libdir=/usr/local/lib | ||
| ``` | ||
| In this example we assume that the boost headers are in `/usr/local/include` while the library files | ||
| are inside `/usr/local/lib`. | ||
|
|
||
| AUTHOR | ||
| ====== | ||
|
|
||
| - __AUTHOR:__ [Vicente Adolfo Bolea Sanchez] [vicente] | ||
| - __AUTHOR:__ [MooHyeon Nam] [mh] | ||
| - __AUTHOR:__ [WonBae Kim] [wb] | ||
| - __AUTHOR:__ [KiBeom Jin] [kb] | ||
| - __AUTHOR:__ [Prof. Nam Beomseok] [nb] | ||
| - __INSTITUTION:__ [DICL laboratory] [dicl] at [UNIST] | ||
|
|
||
| <!-- Links --> | ||
| [vicente]: https://github.com/vicentebolea | ||
| [ym]: https://github.com/youngmoon01 | ||
| [dicl]: http://dicl.unist.ac.kr | ||
| [mh]: https://github.com/nammh | ||
| [wb]: https://github.com/zwigul | ||
| [kb]: https://github.com/kbjin | ||
| [eclipsed]: https://github.com/DICL/eclipsed | ||
| [nb]: http://dicl.unist.ac.kr | ||
| AUTHORS | ||
| ======= | ||
|
|
||
| - __AUTHOR:__ [Vicente Adolfo Bolea Sanchez](http://vicentebolea.me) | ||
| - __AUTHOR:__ [MooHyeon Nam](https://github.com/nammh) | ||
| - __AUTHOR:__ [WonBae Kim](https://github.com/zwigul) | ||
| - __AUTHOR:__ [KiBeom Jin](https://github.com/kbjin) | ||
| - __AUTHOR:__ [Deukyeon Hwang](https://github.com/deukyeon) | ||
| - __AUTHOR:__ [Prof. Nam Beomseok](http://dicl.unist.ac.kr) | ||
| - __INSTITUTION:__ [DICL laboratory](http://dicl.unist.ac.kr) at _UNIST_ | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.