Talend query data in mysql based on inputs from hive -


I need to query a column in the hive and depending on the output I want to query it in MySQL DB.

The flow is something similar to:

  1. Get the list of questions hive tables, IDs
  2. Use the list of IDs to query the MySQL table >

    How do I do this?

    You just need to use a tHiveInput component to retrieve the ID to create your query And then you get two options, the easiest way or the hard (and better) way:

    For easy way you can then run from a tHiveInput component on a tFlowToIterate and then a tmySqlInput component Can iterate Your query in the TMySqlInput component will look something similar:

      "SELECT a, b, c fROM x WHERE id = '" + ((integer) globalMap.get ("row1.id") ) + "'"  

    Line 1 here represents the row running in your TflotOitrate. You can also get this variable by kissing the ctrl + space and selecting the tFlowToIterate_x.id option.

    You can add a tBufferOutput component to your tMySqlInput component to collect all iterations again and then read it with a TuberFinder input component for further processing.

    This is clearly a simple combination that is not the best method for efficiency and is definitely open for SQL injection. But this is a difficult and ready way.

    Your other (hard but potentially better) option is to use a parameterized query using a tMySqlRow component (currently not supported in tmySqlInput component as Tamnand 5.4).

    In Talend, some details about parameterized queries go into the form of a solution of another problem (increasing efficiency) as mentioned in the answer, there is more details on parameterized queries.


Comments

Popular posts from this blog

sqlite3 - UPDATE a table from the SELECT of another one -

c# - Showing a SelectedItem's Property -

javascript - Render HTML after each iteration in loop -