scala - getting number of values within reduceByKey RDD -


When it's called Bike Operation, it's getting a list of the values ​​of a particular key. My question is Is that

  1. Is that the list of values ​​received in sequential order?
  2. Is it possible to know how much value he gets?
  3. I'm trying to calculate the quartile before the list of values ​​of a key, is it possible to reduce bark?

.1 No, it is against the whole situation of completely reducing the operation Going - that means taking advantage of co-operatives and commutitiates to parallel an operation in arbitrary tree of sub-operation.

.2. You will need to define a new monoide by typing an integer monoide and whatever you have to do is assume that your operation is op . 

  yourRdd.map (kv = & gt; (kv._1, (kv._2, 1)). ReduceByKey ((left, right) => (left ._1 op Right ._1, left._2 + right ._2))

You can get a RDD [(KiType, (ReducedValueType, Int))] where Int will be the number of values ​​received for each key.

.3. You have to be more specific about what you mean by quartiles first. Given that 1. There is no answer, you have to be bound by one that defines the first quartile, then you will not need data to sort because you can still filter the values ​​in spite of that.


-

Comments

Post a Comment

Popular posts from this blog

Member with no value in F# -

After I am trying to convert F # to the following C # code: [StructLayout (LayoutKind.Sequential, charset = CharSet.Ansi)] Private class DOCINFOA {[MarshalAs (UnmanagedType.LPStr)] public string nomeDocumento; [Marshall (UnmanagedType.LPStr)] Public string arquivoSaida; [Marshall (UnmanagedType.LPStr)] Public String Tipodo; } I am doing the following: Named place printer module RawPrinterHelper = [& LT; StructLayout (LayoutKind.Sequential, charset = CharSet.Ansi) & gt;] Type DOCINFOA = [& LT; MarshalAs (UnmanagedType.LPStr) & gt;] Member this.nomeDocumento [& LT; MarshalAs (UnmanagedType.LPStr) & gt;] Member this.arquivoSaida [& LT; MarshalAs (UnmanagedType.LPStr) & gt;] Member This Tipoda [/ code> But I get: One type of definition is necessary for one or more members . Can I vacate these members? you val code> member : [& lt; StructLayout (LayoutKind.Sequential, charset = CharSet.Ansi) & gt;] Type DOCINFOA = [...
Read more

java - Joda Time Interval Not returning what I expect -

मेरे पास निम्न प्रोग्राम है import java.util। *; Import java.text। *; आयात करें org.joda.time। *; सार्वजनिक श्रेणी के स्कोप कंट्रोल {सार्वजनिक स्थिर अंतराल मिलनसार () {दिनांक समय currDate = नया दिनांक समय (2008, 4, 4, 15, 30, 0, 0); दिनांक समय epochDate = नया दिनांकटाइम (2000, 1, 1, 12, 0, 0, 0); अंतराल अंतराल = नया अंतराल (युरोप डेट, करोडेट); वापसी अंतराल; } सार्वजनिक स्थिर शून्य मुख्य (स्ट्रिंग [] आर्ग्स) {डबल दिनबात = मिलते समय ()। ToDurationMillis () / 1000/60/60/24; StdOut.println (daysBtween); }} मुझे आउटपुट मिल रहा है: 3016.0 लेकिन जो मैं देख रहा हूं वह है: 3016.1458333333 मैं क्या कर रहा हूँ ? toDurationMillis एक लंबा लौटा देता है, और प्रत्येक प्रभाग को int के रूप में घोषित किया जाता है। जावा इस प्रकार इनट्स को लॉन्ग में परिवर्तित कर देगा और डिवीजन को लंबे समय तक लौटाना होगा। परिणाम को अंत में दोहरे रूप में परिवर्तित करना। डबल्स का उपयोग करके अभिव्यक्ति करने के लिए जावा को बताने के लिए, अभिव्यक्ति के किसी भी घटक को दोहरे रूप में घोषित करें। उदाहरण के लिए: ...
Read more

c# - Showing a SelectedItem's Property -

So I have enabled a rightclick option for my DataGrid. I want to display the only chosen property but it is not behaving like what I would like it to display my name and additional. Public category Paymentinfo {public int PaymentNo {get; Set; } Public string date {get; Set; } Public Double Pay {get; Set; } Public Double Principle {get; Set; } Public Double Interest {get; Set; } Public Double Balance {Received; Set; }} Private Zero MenuItem_OnClick (Object Sender, RoutedEventArgs e) {MessageBox.Show (AmortGrid.SelectedItem.ToString ()); } I'm trying to apply it without using a View Module! If I place a breakpoint where the messagebox and the recipient place it on selected ITEM, then this property will display the payment. A date-payment-principle-interest-balance is the only value that needs payment. Was hoping that it would be something like this MessageBox.Show (AmortGrid.SelectedItem.PaymentNo.ToString ()); When you call ToString () So you get the class typ...
Read more