Skip to content

Commit

Permalink
Updated README and added Hadoop testing script
Browse files Browse the repository at this point in the history
  • Loading branch information
IamMaxim committed Apr 19, 2022
1 parent 3bc7ae5 commit 7509828
Show file tree
Hide file tree
Showing 4 changed files with 38 additions and 26 deletions.
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,5 @@ j2eo.jar
output.eo
output_eo
polystat_check
/hadoop/
/output_hadoop/
43 changes: 17 additions & 26 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@

<img src="https://www.yegor256.com/images/books/elegant-objects/cactus.svg" height="100px" alt="eolang icon"/>

<br>
<br>

[![Gradle Build](https://github.com/polystat/j2eo/actions/workflows/gradle-build.yml/badge.svg)](https://github.com/polystat/j2eo/actions/workflows/gradle-build.yml)
Expand All @@ -19,8 +18,9 @@ This is a translator of **Java** programming language to [EOLANG](https://www.eo
1. Make sure you have installed:
- **Java 16+** (make sure command `java -version` shows 16+ version of Java in terminal if you have multiple Java
version installed)
- **Maven 3.8+** (be aware of [possible conflicts](https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=980467) of the
- **Maven 3.8+** to run tests (be aware of [possible conflicts](https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=980467) of the
latest versions of Maven and Java on some OSs)
- **ANTLR4 4.9.3** (if you want to build the parser on your own. If you don't have ANTLR, you still can build project using bundled version of parser.)
2. Clone the repo into your folder:

HTTPS:
Expand All @@ -37,7 +37,6 @@ This is a translator of **Java** programming language to [EOLANG](https://www.eo
```shell
./build.sh
```
The testing report is generated in the `./j2eo/build/reports/tests/test/index.html` file.
4. After build process, **j2eo.jar** file will appear in the project root folder (`./j2eo`). With this file, is it
possible to translate **.java** files or packages to **.eo** packages. Run:

Expand All @@ -56,6 +55,19 @@ This is a translator of **Java** programming language to [EOLANG](https://www.eo
java -jar j2eo.jar src/test/resources/polystat_tests/test1 -o output_eo
```

### Running translator on Hadoop

Hadoop is a large Java project (contains ~1.8M lines of code as of time of writing this). We included it as a benchmark of the translator.

Repository contains a script to build J2EO, download Hadoop repo and run J2EO on it.

Usage:

```shell
./test-hadoop.sh
```


---

## Motivation
Expand All @@ -73,6 +85,8 @@ particular repository contains translator from Java to EO.
**A**: Publicly available parsers only support older versions of Java, while we aim to support the latest version (
currently 16). Thus, we had to create our own parser.

Also in recent versions, external Java grammar implemented in ANTLR was added as an alternative. It claims to support Java 17, and it does, as for our testing on big projects.

<br>

**Q**: Why do we implement EO AST?
Expand All @@ -89,26 +103,3 @@ bugs in our code. It is also much easier to work with abstraction layer than wit
- First, the Java source code files are parsed recursively.
- Then, for each file, translator converts Java AST to EO AST.
- Then, EO AST is printed out as a source code to output directory in the same directory structure.
---
## NOT covered Java features list
- Type Erasure - Zouev
- Subtyping - discuss with Yegor
- Conversions - remove
- Casting - remove
- Modules
- Exceptions - remove
- Asserts - remove
- Throws - remove
- ``synchronized`` blocks
- ``try``/``catch`` blocks - remove
- ``yeild`` feature
- Threads and Locks
- Generics (all kinds of them) - remove
- Native methods
- break and continue statements - remove
- RTTI (instanceof operator) ?????? - remove
In general, we cover **91 feature of 112** described in the Java language specification.
1 change: 1 addition & 0 deletions build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
set -e

echo "Building J2EO..."
rm -r build/libs
./gradlew fatJar -x test
cp build/libs/* ./
echo "Build completed"
18 changes: 18 additions & 0 deletions test-hadoop.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
#!/usr/bin/env bash

# Exit on error
set -e

echo "Building J2EO..."
./gradlew build -x test jar
cp build/libs/J2EO-0.4.0.jar .

echo "Cloning Hadoop..."
git clone https://github.com/apache/hadoop || true
cd hadoop || exit
echo "Checking out tested revision..."
git checkout ec0ff1dc04b2ced199d71543a8260e9225d9e014
cd ..

echo "Starting J2EO on Hadoop..."
java -jar J2EO-0.4.0.jar -o output_hadoop hadoop

1 comment on commit 7509828

@0pdd
Copy link
Member

@0pdd 0pdd commented on 7509828 Apr 19, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wasn't able to retrieve PDD puzzles from the code base and submit them to GitHub. If you think that it's a bug on our side, please submit it to yegor256/0pdd:

set -x && set -e && set -o pipefail && cd /tmp/0pdd20220303-12-17lwdzx/polystat/j2eo && pdd -v -f /tmp/20220419-3163-1m3kqxt [1]: + set -e + set -o pipefail + cd /tmp/0pdd20220303-12-17lwdzx/polystat/j2eo + pdd -v -f /tmp/20220419-3163-1m3kqxt My version is 0.20.6 Ruby version is 2.6.0 at...

Please, copy and paste this stack trace to GitHub:

UserError
set -x && set -e && set -o pipefail && cd /tmp/0pdd20220303-12-17lwdzx/polystat/j2eo && pdd -v -f /tmp/20220419-3163-1m3kqxt [1]:
+ set -e
+ set -o pipefail
+ cd /tmp/0pdd20220303-12-17lwdzx/polystat/j2eo
+ pdd -v -f /tmp/20220419-3163-1m3kqxt

My version is 0.20.6
Ruby version is 2.6.0 at x86_64-linux
Reading /tmp/0pdd20220303-12-17lwdzx/polystat/j2eo
1127 file(s) found, 67 excluded
/tmp/0pdd20220303-12-17lwdzx/polystat/j2eo/gradle/wrapper/gradle-wrapper.jar is a binary file (59203 bytes)
/tmp/0pdd20220303-12-17lwdzx/polystat/j2eo/src/test/resources/polystat_tests/test1/static_eo/test.eo is a binary file (0 bytes)
Reading .gitignore...
Reading README.md...
Reading .rultor.yml...
Reading settings.gradle.kts...
Reading .dockerignore...
Reading config/checkstyle/checkstyle.xml...
Reading config/xsl/checkstyle-simple.xsl...
Reading .github/workflows/gradle-build.yml...
Reading polystat_check/check.sh...
Reading polystat_check/PolystatTest.java...
Reading gradlew.bat...
Reading test_candidates.sh...
Reading CHANGELOG.md...
Reading gradlew...
Reading gradle/wrapper/gradle-wrapper.properties...
Reading build.gradle.kts...
Reading build.sh...
Reading src/main/resources/stdlib/default.eo...
Reading src/main/resources/stdlib/prim__float.eo...
Reading src/main/resources/stdlib/class__System.eo...
Reading src/main/resources/stdlib/class__Integer.eo...
Reading src/main/resources/stdlib/class__String.eo...
Reading src/main/resources/stdlib/class__PrintStream.eo...
Reading src/main/resources/stdlib/prim__int.eo...
Reading src/main/resources/stdlib/class__Boolean.eo...
Reading src/main/resources/stdlib/prim__number.eo...
Reading src/main/resources/stdlib/prim__boolean.eo...
Reading src/main/resources/stdlib/case.eo...
Reading src/main/resources/stdlib/switch.eo...
Reading src/main/resources/stdlib/class__Float.eo...
Reading src/main/resources/stdlib/class__Object.eo...
Reading src/main/resources/Java_16_Grammar.y...
Reading src/main/java/main/Main2.kt...
Reading src/main/java/eotree/EODot.kt...
Reading src/main/java/eotree/EOBndExpr.kt...
Reading src/main/java/eotree/EOExpr.kt...
Reading src/main/java/eotree/EOMetaObjects.kt...
ERROR: src/main/java/eotree/EOMetaObjects.kt; puzzle at line #15; TODO found, but puzzle can't be parsed, most probably because TODO is not followed by a puzzle marker, as this page explains: https://github.com/yegor256/pdd#how-to-format
If you can't understand the cause of this issue or you don't know how to fix it, please submit a GitHub issue, we will try to help you: https://github.com/yegor256/pdd/issues. This tool is still in its beta version and we will appreciate your feedback. Here is where you can find more documentation: https://github.com/yegor256/pdd/blob/master/README.md.
Exit code is 1

/app/objects/git_repo.rb:66:in `rescue in block in xml'
/app/objects/git_repo.rb:63:in `block in xml'
/app/vendor/ruby-2.6.0/lib/ruby/2.6.0/tempfile.rb:295:in `open'
/app/objects/git_repo.rb:62:in `xml'
/app/objects/puzzles.rb:36:in `deploy'
/app/objects/job.rb:38:in `proceed'
/app/objects/job_starred.rb:33:in `proceed'
/app/objects/job_recorded.rb:32:in `proceed'
/app/objects/job_emailed.rb:35:in `proceed'
/app/objects/job_commiterrors.rb:36:in `proceed'
/app/objects/job_detached.rb:48:in `exclusive'
/app/objects/job_detached.rb:36:in `block in proceed'
/app/objects/job_detached.rb:36:in `fork'
/app/objects/job_detached.rb:36:in `proceed'
/app/0pdd.rb:366:in `block in <top (required)>'
/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1675:in `call'
/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1675:in `block in compile!'
/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1013:in `block (3 levels) in route!'
/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1032:in `route_eval'
/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1013:in `block (2 levels) in route!'
/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1061:in `block in process_route'
/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1059:in `catch'
/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1059:in `process_route'
/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1011:in `block in route!'
/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1008:in `each'
/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1008:in `route!'
/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1129:in `block in dispatch!'
/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1101:in `block in invoke'
/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1101:in `catch'
/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1101:in `invoke'
/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1124:in `dispatch!'
/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:939:in `block in call!'
/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1101:in `block in invoke'
/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1101:in `catch'
/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1101:in `invoke'
/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:939:in `call!'
/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:929:in `call'
/app/vendor/bundle/ruby/2.6.0/gems/rack-protection-2.1.0/lib/rack/protection/xss_header.rb:18:in `call'
/app/vendor/bundle/ruby/2.6.0/gems/rack-protection-2.1.0/lib/rack/protection/path_traversal.rb:16:in `call'
/app/vendor/bundle/ruby/2.6.0/gems/rack-protection-2.1.0/lib/rack/protection/json_csrf.rb:26:in `call'
/app/vendor/bundle/ruby/2.6.0/gems/rack-protection-2.1.0/lib/rack/protection/base.rb:50:in `call'
/app/vendor/bundle/ruby/2.6.0/gems/rack-protection-2.1.0/lib/rack/protection/base.rb:50:in `call'
/app/vendor/bundle/ruby/2.6.0/gems/rack-protection-2.1.0/lib/rack/protection/frame_options.rb:31:in `call'
/app/vendor/bundle/ruby/2.6.0/gems/rack-2.2.3/lib/rack/logger.rb:17:in `call'
/app/vendor/bundle/ruby/2.6.0/gems/rack-2.2.3/lib/rack/common_logger.rb:38:in `call'
/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:253:in `call'
/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:246:in `call'
/app/vendor/bundle/ruby/2.6.0/gems/rack-2.2.3/lib/rack/head.rb:12:in `call'
/app/vendor/bundle/ruby/2.6.0/gems/rack-2.2.3/lib/rack/method_override.rb:24:in `call'
/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:216:in `call'
/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1991:in `call'
/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1542:in `block in call'
/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1769:in `synchronize'
/app/vendor/bundle/ruby/2.6.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1542:in `call'
/app/vendor/bundle/ruby/2.6.0/gems/rack-2.2.3/lib/rack/handler/webrick.rb:95:in `service'
/app/vendor/ruby-2.6.0/lib/ruby/2.6.0/webrick/httpserver.rb:140:in `service'
/app/vendor/ruby-2.6.0/lib/ruby/2.6.0/webrick/httpserver.rb:96:in `run'
/app/vendor/ruby-2.6.0/lib/ruby/2.6.0/webrick/server.rb:307:in `block in start_thread'

Please sign in to comment.