APEXMALHAR-2462 Move JDBC related examples from datatorrent examples to apex-malhar examples #606

prasannapramod · 2017-04-18T20:43:40Z

@tweise @amberarrow @ashwinchandrap please review

prasannapramod · 2017-04-20T04:21:24Z

@tweise @amberarrow @ashwinchandrap i fixed issues please review

prasannapramod · 2017-04-24T16:24:59Z

@tweise can you see

prasannapramod · 2017-04-26T15:00:04Z

@tweise I updated the PR title and commit message. Please see

amberarrow · 2017-04-30T14:34:06Z

Looks good, I'll merge in a couple of days if there are no further comments.

tweise · 2017-04-30T15:15:57Z

examples/jdbc/pom.xml

+    <dependency>
+      <groupId>mysql</groupId>
+      <artifactId>mysql-connector-java</artifactId>
+      <version>5.1.36</version>


There was already a discussion that the mysql connector should not be used due to licensing issue. Why is it required?

The original example is using the mysql jdbc driver and this dependency was probably included so that the apa is complete and can run. Would it be a problem given that example typically is used directly and not as a further dependency to another project.

What would be a good alternative? Make it provided? Use an alternative jdbc driver?

See https://issues.apache.org/jira/browse/APEXMALHAR-2461 - that JIRA needs to be addressed soon and before the next release.

tweise · 2017-04-30T15:18:35Z

examples/jdbc/src/main/resources/META-INF/properties-JdbcToJdbc.xml

+
+  <!-- JDBC driver in use -->
+  <property>
+    <name>dt.application.JdbcToJdbcApp.operator.JdbcInput.prop.store.databaseDriver


application.xxx qualification is not needed here.

tweise · 2017-04-30T15:26:29Z

https://raw.githubusercontent.com/apache/apex-malhar/master/docs/operators/jdbcPollInputOperator.md contains links to old example location. This is an issue with previously moved examples also. I'm OK fixing this outside of this PR if it is addressed soon and before moving any further examples.

tweise · 2017-04-30T15:27:47Z

https://issues.apache.org/jira/browse/APEXMALHAR-2477

tweise · 2017-04-30T15:29:00Z

examples/jdbc/.gitignore

@@ -0,0 +1 @@
+/target/


isn't this taken care of by top level .gitignore ?

tweise · 2017-04-30T15:30:14Z

examples/jdbc/README.md

+    shell> mvn clean install
+
+Upload the `target/jdbcInput-1.0-SNAPSHOT.apa` to the UI console if available or launch it from
+the commandline using `apexcli`.


tweise · 2017-04-30T15:31:55Z

examples/jdbc/README.md

+Step 3: Build the code,
+shell> mvn clean install 
+
+Upload the target/jdbcInput-1.0-SNAPSHOT.apa to the gateway


correct name of the .apa file, remove gateway reference

prasannapramod · 2017-05-27T20:11:43Z

@tweise addressed the comments.

amberarrow · 2017-05-27T23:55:20Z

Looks like jdbc/README.md still has a reference to gateway (line 175) and the line following it (Step 4) is not meaningful, as it currently stands, for users launching from the command line.

The mysql connector reference is still present; how about we address that in a later PR ? One option might be to check if MariaDB connector works as a drop-in replacement and use that if possible. It has LGPL
license.

Rest looks good.

prasannapramod · 2017-05-30T06:53:07Z

@amberarrow @tweise @devtagare please see

tweise · 2017-06-01T05:08:21Z

examples/jdbc/README.md

+Step 3: Build the code,
+shell> mvn clean install 
+
+Upload the target/malhar-examples-JDBC-3.8.0-SNAPSHOT.apa to the gateway


still to be addressed

tweise · 2017-06-01T05:12:12Z

@prasannapramod unless the review comments are addressed, there is no need to ask for another review.

My take is that we should avoid further code additions to Malhar unless the maintenance situation is clear and folks step up to fix issues as they arise (CI errors, dependency issues etc.).

tweise · 2017-06-01T05:14:37Z

examples/jdbc/pom.xml

+  <artifactId>malhar-examples-JDBC</artifactId>
+  <packaging>jar</packaging>
+
+  <!-- change these to the appropriate values -->


remove these lines

tweise · 2017-06-01T05:19:39Z

examples/jdbc/src/main/java/org/apache/apex/examples/JdbcIngest/JdbcHDFSApp.java

+  }
+
+  /**
+   * This method can be modified to have field mappings based on used defined


SPOI-8251 jdbc to jdbc app

prasannapramod · 2017-06-07T18:46:45Z

@amberarrow @tweise Changed references of mysql in README.txt and property files to hsqldb removed "gateway" reference as well.

@tweise Currently there are 3 active example related PR's #609, #604, #606 (current) for fileIO, JMS and Jdbc respectively. I will not rise new example PR's till this are merged and any pending issues you are referring are resolved.

tweise · 2017-06-07T19:02:00Z

@prasannapramod there is follow-up work: package should have been examples.jdbc (please make sure that is consistent for all examples). There is some formatting issue in the README. And I believe there are still issues with user docs pointing to old example locations. Would be good to see pending issues addressed next. Are you going to take up maintenance for all of the example projects?

prasannapramod force-pushed the APEXMALHAR-2462 branch 2 times, most recently from 94a2b79 to f47bf72 Compare April 19, 2017 17:41

prasannapramod force-pushed the APEXMALHAR-2462 branch from f47bf72 to bcc2305 Compare April 24, 2017 18:57

prasannapramod changed the title ~~Apexmalhar 2462 Move JDBC related examples from datatorrent examples to apex-malhar examples~~ Apexmalhar-2462 Move JDBC related examples from datatorrent examples to apex-malhar examples Apr 26, 2017

prasannapramod force-pushed the APEXMALHAR-2462 branch from bcc2305 to e196d28 Compare April 26, 2017 14:17

tweise reviewed Apr 30, 2017

View reviewed changes

examples/jdbc/.gitignore Outdated

@@ -0,0 +1 @@

/target/

Copy link

Contributor

tweise Apr 30, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isn't this taken care of by top level .gitignore ?

tweise reviewed Apr 30, 2017

View reviewed changes

prasannapramod force-pushed the APEXMALHAR-2462 branch 4 times, most recently from 0d6150c to 4f90706 Compare May 25, 2017 00:10

tweise mentioned this pull request May 27, 2017

APEXMALHAR-2468 Move FileIO related examples from datatorrent examples to apex-malhar examples #609

Open

prasannapramod force-pushed the APEXMALHAR-2462 branch from 4f90706 to e8cc278 Compare May 27, 2017 18:54

prasannapramod force-pushed the APEXMALHAR-2462 branch from e8cc278 to 60e01c5 Compare May 30, 2017 06:52

prasannapramod force-pushed the APEXMALHAR-2462 branch 2 times, most recently from 9d50bd9 to e7d4496 Compare May 31, 2017 21:55

prasannapramod changed the title ~~Apexmalhar-2462 Move JDBC related examples from datatorrent examples to apex-malhar examples~~ APEXMALHAR-2462 Move JDBC related examples from datatorrent examples to apex-malhar examples May 31, 2017

prasannapramod closed this May 31, 2017

prasannapramod reopened this May 31, 2017

tweise reviewed Jun 1, 2017

View reviewed changes

YunhanWangUCSD and others added 4 commits June 7, 2017 10:33

SPOI-8252 #resolve #comment Create a simple file to JDBC application

a1b7155

JdbcInput and HDFS output example app

eae0eee

SPOI-8251 jdbc to jdbc app

Fixed tests and POM.Changes related to sql connector.

9b99e0a

APEXMALHAR-2462 Fixed Checkstyle and Log4j Properties.

fc234b1

prasannapramod force-pushed the APEXMALHAR-2462 branch from e7d4496 to fc234b1 Compare June 7, 2017 18:14

asfgit merged commit fc234b1 into apache:master Jun 7, 2017

APEXMALHAR-2462 Move JDBC related examples from datatorrent examples to apex-malhar examples #606

APEXMALHAR-2462 Move JDBC related examples from datatorrent examples to apex-malhar examples #606

Uh oh!

Conversation

prasannapramod commented Apr 18, 2017

Uh oh!

prasannapramod commented Apr 20, 2017

Uh oh!

prasannapramod commented Apr 24, 2017

Uh oh!

prasannapramod commented Apr 26, 2017

Uh oh!

amberarrow commented Apr 30, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tweise commented Apr 30, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tweise commented Apr 30, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

prasannapramod commented May 27, 2017

Uh oh!

amberarrow commented May 27, 2017

Uh oh!

prasannapramod commented May 30, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tweise commented Jun 1, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

prasannapramod commented Jun 7, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tweise commented Jun 7, 2017

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

tweise commented Apr 30, 2017 •

edited

Loading

prasannapramod commented Jun 7, 2017 •

edited

Loading