Skip to content

Commit

Permalink
deploy: 52cf44b
Browse files Browse the repository at this point in the history
  • Loading branch information
pomadchin committed Sep 14, 2023
1 parent f2f61e6 commit 17bb604
Show file tree
Hide file tree
Showing 6 changed files with 20 additions and 20 deletions.
4 changes: 2 additions & 2 deletions Cats.html
Original file line number Diff line number Diff line change
Expand Up @@ -164,7 +164,7 @@ <h2 id="effect-suspension-in-typed-datasets" class="section"><a class="anchor-li
</span><span class="identifier">count</span><span> &lt;- </span><span class="identifier">typedDs</span><span>.</span><span class="identifier">count</span><span>[</span><span class="type-name">Action</span><span>]()
} </span><span class="keyword">yield</span><span> (</span><span class="identifier">sample</span><span>, </span><span class="identifier">count</span><span>)
</span><span class="comment">// result: Action[(Seq[(Int, String)], Long)] = Kleisli(
// cats.data.Kleisli$$Lambda$10814/508592626@5e9eff0d
// cats.data.Kleisli$$Lambda$10740/659879244@786362db
// )</span></code></pre>
<p>As with <code>Job</code>, note that nothing has been run yet. The effect has been properly suspended. To
run our program, we must first supply the <code>SparkSession</code> to the <code>ReaderT</code> layer and then
Expand All @@ -191,7 +191,7 @@ <h3 id="convenience-methods-for-modifying-spark-thread-local-variables" class="s
}
} </span><span class="keyword">yield</span><span> </span><span class="identifier">r</span><span>
</span><span class="comment">// resultWithDescription: Action[(Seq[(Int, String)], Long)] = Kleisli(
// cats.data.Kleisli$$$Lambda$12180/93748074@68de2d92
// cats.data.Kleisli$$$Lambda$12148/604058227@22a8971c
// )
</span><span>
</span><span class="identifier">resultWithDescription</span><span>.</span><span class="identifier">run</span><span>(</span><span class="identifier">spark</span><span>).</span><span class="identifier">unsafeRunSync</span><span>()
Expand Down
2 changes: 1 addition & 1 deletion FeatureOverview.html
Original file line number Diff line number Diff line change
Expand Up @@ -716,7 +716,7 @@ <h2 id="user-defined-functions" class="section"><a class="anchor-link left" href
</span><span class="comment">// priceModifier: (String, Double) =&gt; Double = &lt;function2&gt;
</span><span>
</span><span class="keyword">val</span><span> </span><span class="identifier">udf</span><span> = </span><span class="identifier">aptTypedDs</span><span>.</span><span class="identifier">makeUDF</span><span>(</span><span class="identifier">priceModifier</span><span>)
</span><span class="comment">// udf: (frameless.TypedColumn[Apartment, String], frameless.TypedColumn[Apartment, Double]) =&gt; frameless.TypedColumn[Apartment, Double] = frameless.functions.Udf$$Lambda$13238/2047193583@7f77c3cb
</span><span class="comment">// udf: (frameless.TypedColumn[Apartment, String], frameless.TypedColumn[Apartment, Double]) =&gt; frameless.TypedColumn[Apartment, Double] = frameless.functions.Udf$$Lambda$13179/1429214067@63c1c22c
</span><span>
</span><span class="keyword">val</span><span> </span><span class="identifier">aptds</span><span> = </span><span class="identifier">aptTypedDs</span><span> </span><span class="comment">// For shorter expressions
// aptds: TypedDataset[Apartment] = [city: string, surface: int ... 2 more fields] // For shorter expressions
Expand Down
8 changes: 4 additions & 4 deletions Injection.html
Original file line number Diff line number Diff line change
Expand Up @@ -144,7 +144,7 @@ <h2 id="example" class="section"><a class="anchor-link left" href="#example"><i
</span><span class="comment">// people: Seq[Person] = List(
// Person(
// 42,
// java.util.GregorianCalendar[time=1694437802889,areFieldsSet=true,areAllFieldsSet=true,lenient=true,zone=sun.util.calendar.ZoneInfo[id=&quot;Etc/UTC&quot;,offset=0,dstSavings=0,useDaylight=false,transitions=0,lastRule=null],firstDayOfWeek=1,minimalDaysInFirstWeek=1,ERA=1,YEAR=2023,MONTH=8,WEEK_OF_YEAR=37,WEEK_OF_MONTH=3,DAY_OF_MONTH=11,DAY_OF_YEAR=254,DAY_OF_WEEK=2,DAY_OF_WEEK_IN_MONTH=2,AM_PM=1,HOUR=1,HOUR_OF_DAY=13,MINUTE=10,SECOND=2,MILLISECOND=889,ZONE_OFFSET=0,DST_OFFSET=0]
// java.util.GregorianCalendar[time=1694700311782,areFieldsSet=true,areAllFieldsSet=true,lenient=true,zone=sun.util.calendar.ZoneInfo[id=&quot;Etc/UTC&quot;,offset=0,dstSavings=0,useDaylight=false,transitions=0,lastRule=null],firstDayOfWeek=1,minimalDaysInFirstWeek=1,ERA=1,YEAR=2023,MONTH=8,WEEK_OF_YEAR=37,WEEK_OF_MONTH=3,DAY_OF_MONTH=14,DAY_OF_YEAR=257,DAY_OF_WEEK=5,DAY_OF_WEEK_IN_MONTH=2,AM_PM=1,HOUR=2,HOUR_OF_DAY=14,MINUTE=5,SECOND=11,MILLISECOND=782,ZONE_OFFSET=0,DST_OFFSET=0]
// )
// )</span></code></pre>
<p>And an instance of a <code>TypedDataset</code>:</p>
Expand All @@ -167,7 +167,7 @@ <h2 id="example" class="section"><a class="anchor-link left" href="#example"><i
</span><span class="identifier">cal</span><span>
}
}
</span><span class="comment">// calendarToLongInjection: AnyRef with Injection[Calendar, Long] = repl.MdocSession$MdocApp0$$anon$1@1724577</span></code></pre>
</span><span class="comment">// calendarToLongInjection: AnyRef with Injection[Calendar, Long] = repl.MdocSession$MdocApp0$$anon$1@30b549ab</span></code></pre>
<p>We can be less verbose using the <code>Injection.apply</code> function:</p>
<pre class="keep-together pdf epub"><code class="nohighlight"><span class="keyword">import</span><span> </span><span class="identifier">frameless</span><span>.</span><span class="identifier">_</span><span>

Expand All @@ -180,7 +180,7 @@ <h2 id="example" class="section"><a class="anchor-link left" href="#example"><i
</span><span class="identifier">cal</span><span>.</span><span class="identifier">setTime</span><span>(</span><span class="keyword">new</span><span> </span><span class="identifier">java</span><span>.</span><span class="identifier">util</span><span>.</span><span class="type-name">Date</span><span>(</span><span class="identifier">l</span><span>))
</span><span class="identifier">cal</span><span>
})
</span><span class="comment">// calendarToLongInjection: Injection[Calendar, Long] = frameless.Injection$$anon$1@3abe27bc</span></code></pre>
</span><span class="comment">// calendarToLongInjection: Injection[Calendar, Long] = frameless.Injection$$anon$1@341fb7da</span></code></pre>
<p>Now we can create our <code>TypedDataset</code>:</p>
<pre class="keep-together pdf epub"><code class="nohighlight"><span class="keyword">val</span><span> </span><span class="identifier">personDS</span><span> = </span><span class="type-name">TypedDataset</span><span>.</span><span class="identifier">create</span><span>(</span><span class="identifier">people</span><span>)
</span><span class="comment">// personDS: TypedDataset[Person] = [age: int, birthday: bigint]</span></code></pre>
Expand Down Expand Up @@ -214,7 +214,7 @@ <h2 id="another-example" class="section"><a class="anchor-link left" href="#anot
</span><span class="keyword">case</span><span> </span><span class="number-literal">2</span><span> =&gt; </span><span class="type-name">Female</span><span>
</span><span class="keyword">case</span><span> </span><span class="number-literal">3</span><span> =&gt; </span><span class="type-name">Other</span><span>
})
</span><span class="comment">// genderToInt: Injection[Gender, Int] = frameless.Injection$$anon$1@4e927fb5</span></code></pre>
</span><span class="comment">// genderToInt: Injection[Gender, Int] = frameless.Injection$$anon$1@b60eb2e</span></code></pre>
<p>And now we can create our <code>TypedDataset</code>:</p>
<pre class="keep-together pdf epub"><code class="nohighlight"><span class="keyword">val</span><span> </span><span class="identifier">personDS</span><span> = </span><span class="type-name">TypedDataset</span><span>.</span><span class="identifier">create</span><span>(</span><span class="identifier">people</span><span>)
</span><span class="comment">// personDS: TypedDataset[Person] = [age: int, gender: int]</span></code></pre>
Expand Down
4 changes: 2 additions & 2 deletions Job.html
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,7 @@ <h1 id="job-a" class="title">Job[A]</h1>
</span><span class="identifier">count</span><span> &lt;- </span><span class="identifier">ds</span><span>.</span><span class="identifier">count</span><span>()
</span><span class="identifier">sample</span><span> &lt;- </span><span class="identifier">ds</span><span>.</span><span class="identifier">take</span><span>((</span><span class="identifier">count</span><span>/</span><span class="number-literal">5</span><span>).</span><span class="identifier">toInt</span><span>)
} </span><span class="keyword">yield</span><span> </span><span class="identifier">sample</span><span>
</span><span class="comment">// countAndTakeJob: frameless.Job[Seq[Int]] = frameless.Job$$anon$3@13359667
</span><span class="comment">// countAndTakeJob: frameless.Job[Seq[Int]] = frameless.Job$$anon$3@5b6fa076
</span><span>
</span><span class="identifier">countAndTakeJob</span><span>.</span><span class="identifier">run</span><span>()
</span><span class="comment">// res1: Seq[Int] = WrappedArray(1, 2, 3, 4)</span></code></pre>
Expand All @@ -165,7 +165,7 @@ <h1 id="job-a" class="title">Job[A]</h1>
</span><span class="keyword">def</span><span> </span><span class="declaration-name">computeMinOfSample</span><span>(</span><span class="identifier">sample</span><span>: </span><span class="type-name">Job</span><span>[</span><span class="type-name">Seq</span><span>[</span><span class="type-name">Int</span><span>]]): </span><span class="type-name">Job</span><span>[</span><span class="type-name">Int</span><span>] = </span><span class="identifier">sample</span><span>.</span><span class="identifier">map</span><span>(</span><span class="identifier">_</span><span>.</span><span class="identifier">min</span><span>)

</span><span class="keyword">val</span><span> </span><span class="identifier">finalJob</span><span> = </span><span class="identifier">computeMinOfSample</span><span>(</span><span class="identifier">countAndTakeJob</span><span>)
</span><span class="comment">// finalJob: Job[Int] = frameless.Job$$anon$2@671b8c9e</span></code></pre>
</span><span class="comment">// finalJob: Job[Int] = frameless.Job$$anon$2@f50b697</span></code></pre>
<p>Now we can execute this new job by specifying a <a href="https://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.SparkContext@setJobGroup(groupId:String,description:String,interruptOnCancel:Boolean):Unit">group-id</a> and a description.
This allows the programmer to see this information on the Spark UI and help track, say,
performance issues.</p>
Expand Down
2 changes: 1 addition & 1 deletion TypedEncoder.html
Original file line number Diff line number Diff line change
Expand Up @@ -210,7 +210,7 @@ <h1 id="typed-encoders-in-frameless" class="title">Typed Encoders in Frameless</
</span><span class="comment">// ds: TypedDataset[Foo] = [i: int, b: struct&lt;d: double, s: string&gt;]
</span><span>
</span><span class="identifier">ds</span><span>.</span><span class="identifier">collect</span><span>()
</span><span class="comment">// res3: frameless.Job[Seq[Foo]] = frameless.Job$$anon$4@1ec9e3a9</span></code></pre>
</span><span class="comment">// res3: frameless.Job[Seq[Foo]] = frameless.Job$$anon$4@3b30a39e</span></code></pre>
<p>But any non-encodable in the case class hierarchy will be detected at compile time:</p>
<pre class="keep-together pdf epub"><code class="nohighlight"><span class="keyword">case</span><span> </span><span class="keyword">class</span><span> </span><span class="type-name">BarDate</span><span>(</span><span class="identifier">d</span><span>: </span><span class="type-name">Double</span><span>, </span><span class="identifier">s</span><span>: </span><span class="type-name">String</span><span>, </span><span class="identifier">t</span><span>: </span><span class="identifier">java</span><span>.</span><span class="identifier">util</span><span>.</span><span class="type-name">Calendar</span><span>)
</span><span class="keyword">case</span><span> </span><span class="keyword">class</span><span> </span><span class="type-name">FooDate</span><span>(</span><span class="identifier">i</span><span>: </span><span class="type-name">Int</span><span>, </span><span class="identifier">b</span><span>: </span><span class="type-name">BarDate</span><span>)</span></code></pre>
Expand Down
Loading

0 comments on commit 17bb604

Please sign in to comment.