forked from D-Net/dnet-hadoop
removed the repartition(1). Added code for the creation of an archive containing all the parts dumped for each community
This commit is contained in:
parent
1991a49f70
commit
341dc59ead
|
@ -59,7 +59,6 @@ public class CommunitySplit implements Serializable {
|
|||
try {
|
||||
community_products.first();
|
||||
community_products
|
||||
.repartition(1)
|
||||
.write()
|
||||
.option("compression", "gzip")
|
||||
.mode(SaveMode.Overwrite)
|
||||
|
|
Loading…
Reference in New Issue