Scala to Python - sparkSql folder #2
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 6 commits
September 30, 2017 12:21
| @@ -1,5 +1,5 @@ | |||
| import re | |||
|
|
|||
| class Utils(): | |||
|
|
|||
| COMMA_DELIMITER = re.compile(''',(?=(?:[^'"]|'[^']*'|"[^"]*")*$)''') | |||
There was a problem hiding this comment.
why do we need to make this small change?
There was a problem hiding this comment.
I made some changes to the original one in scala to run in python (because the languages parse regex in a different way), and among those changes I, mistakenly, made the new one not match commas within single quotations too. But this raised some problems when processing the stackoverflow survey. I just removed that.
jleetutorial
approved these changes
Sep 30, 2017
evross
pushed a commit
to evross/python-spark-tutorial
that referenced
this pull request
Dec 3, 2018
* Add env/ and .vscode/ to .gitignore * Add IAM requirements.txt * Python Service account keys (jleetutorial#5) Add IAM service account keys snippets * Python Quickstart (jleetutorial#3) Add IAM quickstart * Python Service accounts (jleetutorial#4) Add IAM service account snippets * Add query grantable roles and tests (jleetutorial#2) * Add whitespace and correct string formatting * Add extra CR * Lint
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Small change to regex on commons/Utils.py
Converted the scala files on the sparkSql folder to python
Ignored sparkSql/TypedDataset.scala
Changed sparkSql/RddDatasetConversion to sparkSql/RddDataframeConversion