Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add cypher AST #250

Open
wants to merge 9 commits into
base: master
Choose a base branch
from
Open

add cypher AST #250

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
7707bfb
add feature cypher ast
thamindumk Aug 16, 2024
71c8349
add feature cypher ast
thamindumk Aug 17, 2024
6878eb9
add feature cypher ast
thamindumk Aug 17, 2024
79ec206
Merge pull request #1 from thamindumk/feature/ast
thamindumk Aug 17, 2024
ab5e605
add license and print ast in logger
thamindumk Aug 23, 2024
71f3cbe
Merge pull request #2 from thamindumk/feature/ast
thamindumk Aug 23, 2024
6a86ee4
fix issue on a TODO comment
thamindumk Aug 28, 2024
516cd42
Merge pull request #3 from thamindumk/feature/ast
thamindumk Aug 28, 2024
bf32c76
Merge pull request #5 from miyurud/master
thamindumk Nov 15, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 17 additions & 2 deletions CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
cmake_minimum_required(VERSION 3.10)
project(JasmineGraph)
set(CMAKE_CXX_STANDARD 11)
set(CMAKE_CXX_STANDARD 17)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We use C++ 11 standard currently in JasmineGraph. Is there special reason to change the C++ standard to 17?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sir, I have to use "any_cast" and "std::any" in the cypher ast, because I have to override the methods of generated parse by antlr4 and those methods use that "std::any". it is a utility which is provided by c++ 17 standards. that's why I used C++ 17 standards.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not a simple change. We have to evaluate the effect of this for the entire project. We should not do standard upgrade unless we assess the essential requirement of doing so, what would be the complications, and the development workload attached to that. So what you are saying is Cypher is dependent on C++ 17?


set(THREADS_PREFER_PTHREAD_FLAG ON)
find_package(Threads REQUIRED)
Expand Down Expand Up @@ -39,6 +39,11 @@ set(HEADERS globals.h
src/query/algorithms/linkprediction/JasminGraphLinkPredictor.h
src/query/algorithms/triangles/Triangles.h
src/query/algorithms/triangles/StreamingTriangles.h
src/query/processor/cypher/astbuilder/ASTBuilder.h
src/query/processor/cypher/astbuilder/ASTInternalNode.h
src/query/processor/cypher/astbuilder/ASTLeafNoValue.h
miyurud marked this conversation as resolved.
Show resolved Hide resolved
src/query/processor/cypher/astbuilder/ASTNode.h
src/query/processor/cypher/astbuilder/ASTLeafValue.h
src/scale/scaler.h
src/server/JasmineGraphInstance.h
src/server/JasmineGraphInstanceFileTransferService.h
Expand Down Expand Up @@ -106,6 +111,11 @@ set(SOURCES src/backend/JasmineGraphBackend.cpp
src/query/algorithms/linkprediction/JasminGraphLinkPredictor.cpp
src/query/algorithms/triangles/Triangles.cpp
src/query/algorithms/triangles/StreamingTriangles.cpp
src/query/processor/cypher/astbuilder/ASTBuilder.cpp
src/query/processor/cypher/astbuilder/ASTInternalNode.cpp
src/query/processor/cypher/astbuilder/ASTLeafNoValue.cpp
src/query/processor/cypher/astbuilder/ASTNode.cpp
src/query/processor/cypher/astbuilder/ASTLeafValue.cpp
src/scale/scaler.cpp
src/server/JasmineGraphInstance.cpp
src/server/JasmineGraphInstanceFileTransferService.cpp
Expand Down Expand Up @@ -137,7 +147,8 @@ if (CMAKE_BUILD_TYPE STREQUAL "DEBUG")
add_compile_options(-DUNIT_TEST)
endif ()

add_library(JasmineGraphLib ${HEADERS} ${SOURCES})
file(GLOB GENERATED_SRC ${CMAKE_SOURCE_DIR}/code_generated/antlr/*.cpp)
add_library(JasmineGraphLib ${HEADERS} ${SOURCES} ${GENERATED_SRC})
add_executable(JasmineGraph main.h main.cpp)

target_compile_definitions(JasmineGraphLib PUBLIC ROOT_DIR="${CMAKE_CURRENT_SOURCE_DIR}/")
Expand All @@ -162,11 +173,15 @@ target_link_libraries(JasmineGraphLib PRIVATE /usr/local/lib/libcppkafka.so)
target_link_libraries(JasmineGraph JasmineGraphLib)
target_link_libraries(JasmineGraph curl)

include_directories(/usr/local/include/antlr4-runtime)
link_directories(/usr/local/lib)
include_directories(/usr/local/include/yaml-cpp)
target_link_libraries(JasmineGraphLib PRIVATE m)
target_link_libraries(JasmineGraphLib PRIVATE /usr/local/lib/libkubernetes.so)
target_link_libraries(JasmineGraphLib PRIVATE yaml-cpp)
target_link_libraries(JasmineGraphLib PRIVATE curl)
target_link_libraries(JasmineGraphLib PRIVATE antlr4-runtime)


if (CMAKE_BUILD_TYPE STREQUAL "DEBUG")
# Include google test
Expand Down
17 changes: 17 additions & 0 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,23 @@ RUN if [ "$DEBUG" = "true" ]; then apt-get update \
&& apt-get install --no-install-recommends -y gdb gdbserver \
&& apt-get clean; fi

WORKDIR "${HOME}"
RUN mkdir antlr
WORKDIR "${HOME}"/antlr
RUN apt-get update && apt-get install --no-install-recommends -y default-jre
RUN curl -O https://s3.amazonaws.com/artifacts.opencypher.org/M23/Cypher.g4
RUN curl -O https://www.antlr.org/download/antlr-4.13.2-complete.jar
RUN java -jar antlr-4.13.2-complete.jar -Dlanguage=Cpp -visitor Cypher.g4
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we move this entire process of downloading, installing, running, and removing the JVM to pre-requisites image? I do not see a point of doing that here.

RUN apt-get purge default-jre -y

WORKDIR "${JASMINEGRAPH_HOME}"
RUN mkdir "${JASMINEGRAPH_HOME}"/code_generated/
RUN mkdir "${JASMINEGRAPH_HOME}"/code_generated/antlr
RUN mv /home/ubuntu/antlr/*.cpp "${JASMINEGRAPH_HOME}"/code_generated/antlr
RUN mv /home/ubuntu/antlr/*.h "${JASMINEGRAPH_HOME}"/code_generated/antlr
RUN rm -f "${HOME}"/antlr/*

WORKDIR "${JASMINEGRAPH_HOME}"
COPY ./build.sh ./build.sh
COPY ./CMakeLists.txt ./CMakeLists.txt
COPY ./main.h ./main.h
Expand Down
53 changes: 51 additions & 2 deletions src/frontend/JasmineGraphFrontEnd.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,12 @@ limitations under the License.
#include "JasmineGraphFrontEndProtocol.h"
#include "core/CoreConstants.h"
#include "core/scheduler/JobScheduler.h"
#include "antlr4-runtime.h"
#include "/home/ubuntu/software/jasminegraph/code_generated/antlr/CypherLexer.h"
#include "/home/ubuntu/software/jasminegraph/code_generated/antlr/CypherParser.h"
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than specifying absolute path here, we should use relative path instead.

#include "../query/processor/cypher/astbuilder/ASTBuilder.h"
#include "../query/processor/cypher/astbuilder/ASTNode.h"


#define MAX_PENDING_CONNECTIONS 10
#define DATA_BUFFER_SIZE (FRONTEND_DATA_LENGTH + 1)
Expand All @@ -64,6 +70,7 @@ bool JasmineGraphFrontEnd::strian_exit;

static std::string getPartitionCount(std::string path);
static void list_command(int connFd, SQLiteDBInterface *sqlite, bool *loop_exit_p);
static void cypher_ast_command(int connFd, bool *loop_exit_p);
static void add_rdf_command(std::string masterIP, int connFd, SQLiteDBInterface *sqlite, bool *loop_exit_p);
static void add_graph_command(std::string masterIP, int connFd, SQLiteDBInterface *sqlite, bool *loop_exit_p);
static void add_graph_cust_command(std::string masterIP, int connFd, SQLiteDBInterface *sqlite, bool *loop_exit_p);
Expand Down Expand Up @@ -169,7 +176,9 @@ void *frontendservicesesion(void *dummyPt) {
break;
} else if (line.compare(LIST) == 0) {
list_command(connFd, sqlite, &loop_exit);
} else if (line.compare(SHTDN) == 0) {
} else if (line.compare(CYPHER_AST) == 0){
cypher_ast_command(connFd, &loop_exit);
}else if (line.compare(SHTDN) == 0) {
JasmineGraphServer::shutdown_workers();
close(connFd);
exit(0);
Expand Down Expand Up @@ -627,6 +636,45 @@ static void list_command(int connFd, SQLiteDBInterface *sqlite, bool *loop_exit_
}
}

static void cypher_ast_command(int connFd, bool *loop_exit)
{

string msg_1 = "Input Query :";
int result_wr = write(connFd, msg_1.c_str(), msg_1.length());
if (result_wr < 0) {
frontend_logger.error("Error writing to socket");
*loop_exit = true;
return;
}
result_wr = write(connFd, "\r\n", 2);
if (result_wr < 0) {
frontend_logger.error("Error writing to socket");
*loop_exit = true;
return;
}

// Get user response.
char user_res[FRONTEND_DATA_LENGTH + 1];
bzero(user_res, FRONTEND_DATA_LENGTH + 1);
read(connFd, user_res, FRONTEND_DATA_LENGTH);
string user_res_s(user_res);

antlr4::ANTLRInputStream input(user_res_s);
// Create a lexer from the input
CypherLexer lexer(&input);

// Create a token stream from the lexer
antlr4::CommonTokenStream tokens(&lexer);

// Create a parser from the token stream
CypherParser parser(&tokens);

ASTBuilder ast_builder;
auto* ast = any_cast<ASTNode*>(ast_builder.visitOC_Cypher(parser.oC_Cypher()));
string result = ast->print(1);
frontend_logger.log(result, "log");
}

static void add_rdf_command(std::string masterIP, int connFd, SQLiteDBInterface *sqlite, bool *loop_exit_p) {
// add RDF graph
int result_wr = write(connFd, SEND.c_str(), FRONTEND_COMMAND_LENGTH);
Expand Down Expand Up @@ -804,7 +852,8 @@ static void add_graph_command(std::string masterIP, int connFd, SQLiteDBInterfac
int result_wr = write(connFd, DONE.c_str(), DONE.size());
if (result_wr < 0) {
frontend_logger.error("Error writing to socket");
*loop_exit_p = true;
*loop_exit_p =
true;
return;
}
result_wr = write(connFd, "\r\n", 2);
Expand Down
1 change: 1 addition & 0 deletions src/frontend/JasmineGraphFrontEndProtocol.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,4 +54,5 @@ const string SLA = "sla";
const string COMMAND = "command";
const string PRIORITY = "priority(>=1)";
const string INVALID_FORMAT = "Invalid message format";
const string CYPHER_AST = "cypher-ast";

1 change: 1 addition & 0 deletions src/frontend/JasmineGraphFrontEndProtocol.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,7 @@ extern const string STOP_STRIAN;
extern const string ADMDL;
extern const string MERGE;
extern const string INVALID_FORMAT;
extern const string CYPHER_AST;

class JasminGraphFrontEndProtocol {
// Note that this protocol do not need a handshake session since the communication in most of the time is conducted
Expand Down
Loading