Using Hadoop through a SOCKS proxy? -


Then our Hadoop cluster runs on some nodes and can only be reached from these nodes. You do SSH and your work.

Since it is quite upset, but (obviously) nobody will be able to go to try to configure access control so that it is useful for some outside, I try the next best thing Use SSH to run a sock proxy in the cluster, i.e.: 

  $ ssh -D localhost: 10000 the.gateway cat

There are whispers of SOCKS support (naturally I have not received any document), and go Ira as it core-site.xml : 

  & lt; Property & gt; & Lt; Name & gt; Fs.default.name & lt; / Name & gt; & Lt; Price & gt; HDFS: //reachable.from.behind.proxy: 1234 / & lt; / Value & gt; & Lt; / Property & gt; & Lt; Property & gt; & Lt; Name & gt; Mapred.job.tracker & lt; / Name & gt; & Lt; Price & gt; Reachable.from.behind.proxy: 5678 & lt; / Value & gt; & Lt; / Property & gt; & Lt; Property & gt; & Lt; Name & gt; Hadoop.rpc.socket.factory.class.default & lt; / Name & gt; & Lt; Price & gt; Org.apache.hadoop.net.SocksSocketFactory & lt; / Value & gt; & Lt; / Property & gt; & Lt; Property & gt; & Lt; Name & gt; Hadoop.socks.server & lt; / Name & gt; & Lt; Price & gt; Local Host: 10000 & lt; / Value & gt; & Lt; / Property & gt;  
  
  
  
  
  
  
 < / P> 
 
 I am just trying to run jobs, not administering cluster. I only need to access HDFS and deposit jobs through SDOS (SOCS) through SOCS (It seems that there is a completely different thing about using SSL / proxy between cluster nodes etc.; It should not be that my machine is not part of a cluster, just a customer.)  
 Is there any useful document on that? To illustrate my failure to do something useful: I am through  strace -f . We found the configuration value by running the client and examining the configuration files to be read. 
 
 Out of which is the description of the configuration value, even its response? (I've got a really void reference document, just a different old tutorial, I hope I'm missing something?) 
 
 Is there really any way to dump the configuration value that it uses? is?

The original code for this to be applied was added to

but this article Also notes that you have to change the socket class with socks

with

& lt; Property & gt; & Lt; Name & gt; Hadoop.rpc.socket.factory.class.default & lt; / Name & gt; & Lt; Price & gt; Org.apache.hadoop.net.SocksSocketFactory & lt; / Pricing & gt; & Lt; / Property & gt; Edit: Note that the properties are in different files:

  1. fs.default.name and hadoop.socks.server and hadoop.rpc.socket Factory.class.default is cor- site.xml
  2. mapped. JobTracker and Mapped Job Tractor. Mapred-site.xml (config for map-less)
need to go to HTTP.addressconfig
-

Comments

Post a Comment

Popular posts from this blog

Member with no value in F# -

After I am trying to convert F # to the following C # code: [StructLayout (LayoutKind.Sequential, charset = CharSet.Ansi)] Private class DOCINFOA {[MarshalAs (UnmanagedType.LPStr)] public string nomeDocumento; [Marshall (UnmanagedType.LPStr)] Public string arquivoSaida; [Marshall (UnmanagedType.LPStr)] Public String Tipodo; } I am doing the following: Named place printer module RawPrinterHelper = [& LT; StructLayout (LayoutKind.Sequential, charset = CharSet.Ansi) & gt;] Type DOCINFOA = [& LT; MarshalAs (UnmanagedType.LPStr) & gt;] Member this.nomeDocumento [& LT; MarshalAs (UnmanagedType.LPStr) & gt;] Member this.arquivoSaida [& LT; MarshalAs (UnmanagedType.LPStr) & gt;] Member This Tipoda [/ code> But I get: One type of definition is necessary for one or more members . Can I vacate these members? you val code> member : [& lt; StructLayout (LayoutKind.Sequential, charset = CharSet.Ansi) & gt;] Type DOCINFOA = [...
Read more

java - Joda Time Interval Not returning what I expect -

मेरे पास निम्न प्रोग्राम है import java.util। *; Import java.text। *; आयात करें org.joda.time। *; सार्वजनिक श्रेणी के स्कोप कंट्रोल {सार्वजनिक स्थिर अंतराल मिलनसार () {दिनांक समय currDate = नया दिनांक समय (2008, 4, 4, 15, 30, 0, 0); दिनांक समय epochDate = नया दिनांकटाइम (2000, 1, 1, 12, 0, 0, 0); अंतराल अंतराल = नया अंतराल (युरोप डेट, करोडेट); वापसी अंतराल; } सार्वजनिक स्थिर शून्य मुख्य (स्ट्रिंग [] आर्ग्स) {डबल दिनबात = मिलते समय ()। ToDurationMillis () / 1000/60/60/24; StdOut.println (daysBtween); }} मुझे आउटपुट मिल रहा है: 3016.0 लेकिन जो मैं देख रहा हूं वह है: 3016.1458333333 मैं क्या कर रहा हूँ ? toDurationMillis एक लंबा लौटा देता है, और प्रत्येक प्रभाग को int के रूप में घोषित किया जाता है। जावा इस प्रकार इनट्स को लॉन्ग में परिवर्तित कर देगा और डिवीजन को लंबे समय तक लौटाना होगा। परिणाम को अंत में दोहरे रूप में परिवर्तित करना। डबल्स का उपयोग करके अभिव्यक्ति करने के लिए जावा को बताने के लिए, अभिव्यक्ति के किसी भी घटक को दोहरे रूप में घोषित करें। उदाहरण के लिए: ...
Read more

c# - Showing a SelectedItem's Property -

So I have enabled a rightclick option for my DataGrid. I want to display the only chosen property but it is not behaving like what I would like it to display my name and additional. Public category Paymentinfo {public int PaymentNo {get; Set; } Public string date {get; Set; } Public Double Pay {get; Set; } Public Double Principle {get; Set; } Public Double Interest {get; Set; } Public Double Balance {Received; Set; }} Private Zero MenuItem_OnClick (Object Sender, RoutedEventArgs e) {MessageBox.Show (AmortGrid.SelectedItem.ToString ()); } I'm trying to apply it without using a View Module! If I place a breakpoint where the messagebox and the recipient place it on selected ITEM, then this property will display the payment. A date-payment-principle-interest-balance is the only value that needs payment. Was hoping that it would be something like this MessageBox.Show (AmortGrid.SelectedItem.PaymentNo.ToString ()); When you call ToString () So you get the class typ...
Read more