Spark SQL filter equal values

Standard

When filtering in a DataFrame in Scala you need to use a triple = (i.e. ===) to filter:

So you must use:

ss.filter(ss("ProductKey") === 68325).show()

Because using

ss.filter(ss("ProductKey") == 68325).show()

will return the following error:

<console>:25: error: overloaded method value filter with alternatives:
  (conditionExpr: String)org.apache.spark.sql.DataFrame <and>
  (condition: org.apache.spark.sql.Column)org.apache.spark.sql.DataFrame
 cannot be applied to (Boolean)
              ss.filter(ss("ProductKey") == 68325).show()

Also

ss.filter(ss("ProductKey") = 68325).show()

will return the following error as you cannot update a DataFrame

<console>:25: error: value update is not a member of org.apache.spark.sql.DataFrame
              ss.filter(ss("StopSaleOnPropertyKey") = 68325).show()

 

Advertisements

2 thoughts on “Spark SQL filter equal values

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s