Questions tagged [mappers]
25 questions
18
votes
1 answer
Reusing MyBatis ResultMap in multiple mapper.xml
I would like to re-use a specific from different *Mapper.xml files which all somehow read same objects.
I have a Database table called Project, which I created the following resultMap for:
…
Alex Paransky
- 295
- 1
- 3
- 12
4
votes
1 answer
mapreduce conditional probability
how to do probability aggregations inside my reducer with mappers;
I'm trying to implement both the “stripes” approach and the “pair” approach on Hadoop for the following tasks but I'd like to know how to do communication among multiple mappers and…
user3448593
- 49
- 2
3
votes
0 answers
Why combiner input records are more than mapper output records?
Combiner works on output records of mapper. If the mapper output records are fed to the combiner then why are my combiner input records are more than mapper output records?
I got these 80 records extra.I have no idea from where they came & what…
shriyog
- 938
- 1
- 13
- 26
3
votes
2 answers
hadoop get actual number of mappers
In the map phase of my program, I need to know the total number of mappers that are created. This will help me in the key creation process of the map (I want to emit as many key-value pairs for each object as the number of mappers).
I know that…
vefthym
- 7,422
- 6
- 32
- 58
3
votes
3 answers
Setting the number of Reducers for an Amazon EMR application
I am trying to run the wordcount example under Amazon EMR.
-1- First, I create a cluster with the following command:
./elastic-mapreduce --create --name "MyTest" --alive
This creates a cluster with a single instance and returns a jobID, lets say…
user2262938
- 81
- 4
3
votes
1 answer
Will Hadoop always spread the work evenly between nodes?
Say I have 200 input files and 20 nodes, and each node has 10 mapper slots. Will Hadoop always allocate the work evenly, such that each node will get 10 input files and simultaneously start 10 mappers? Is there a way to force this behavior?
sangfroid
- 3,733
- 11
- 38
- 42
2
votes
1 answer
Java - Orika 1.4.5 - OutOfMemoryException : PermGen Space
I have a problem with Orika 1.4.5 and PermGen Space.
Indeed, i'm using a a ConfigurableMapper this way :
public class SoapSearchPrdvFreeSlotsMapper extends ConfigurableMapper {
@Override
public void configure(MapperFactory mapperFactory) {
…
Psychose
- 23
- 4
2
votes
1 answer
With Hadoop, how to change the number of mappers for a given job?
So, I have two jobs, Job A and Job B. For Job A, I would like to have a maximum of 6 mappers per node. However, Job B is a little different. For Job B, I can only run one mapper per node. The reason for this isn't important -- let's just say…
sangfroid
- 3,733
- 11
- 38
- 42
1
vote
4 answers
Mappers, Reducers, FIlters
I know about map/reduce alghoritm and its use. It's using functions that are called Mappers and Reducers, but I also find people use the word Filters.
Are Filters same as Mappers or is there some significant difference?
Azder
- 4,698
- 7
- 37
- 57
1
vote
1 answer
How to set the number of NameNodes, DataNodes, Mappers and Reducers in Hadoop
I want to know how to set the number of
NameNodes
DataNodes
Mappers
Reducers
in code/configuration of Hadoop.
Mosab Shaheen
- 1,114
- 10
- 25
1
vote
1 answer
MyBatis+Spring MapperScan with Mulitple Data Sources
I am pulling data from two different databases using MyBatis 3.3.1 and Spring 4.3. The two configuration classes to scan for mappers look at follows:
@Configuration
@MapperScan(value="com.mapper1.map",
…
tux
- 55
- 1
- 5
1
vote
2 answers
Cytoscape.js mapData mapper returning same color for all vertices
I am trying to use the Cytoscape.js mapData() mapper to produce a color gradient between red and blue for my vertice's background color. I am using the exact example from the documentation:
'background-color': 'mapData(degree, 1, 100, blue,…
M. Holdefer
- 11
- 1
1
vote
1 answer
Ant Script for converting multiple file formats using mappers- creating double extension files
I want to convert multiple file formats to a single file format.
Example: D:\myrepo\rough has 3 files
1. abc.sql
2. def.xml
3. ghi.dmp
I want them all to be converted to .txt using glob mappers.
user
- 67
- 1
- 6
1
vote
1 answer
Cast an object inside lambda expressions
i'm using EntityTypeConfiguration to map my database.
The problem is, the class T_DOC_GENERIC inherits T_DOC, when I set my relationship WithMany he expects an object T_DOC_GENERIC which he's declared as T_DOC.
public class T_DOC_GENERICMapper :…
Pedro
- 25
- 5
1
vote
1 answer
Setting Mappers of desired numbers
I have gone through lot of blogs on stackoverflow and also apache wiki for getting to know the way the mappers are set in Hadoop. I also went through [hadoop - how total mappers are determined [this] post.
Some say its based on InputFormat and some…
user3560220
- 221
- 3
- 4
- 11