Mincemeat (mincemeat.py) is a Python implementation of the MapReduce framework.
mincemeat.py is a lightweight, fault tolerant, secure, open source Python implementation of the MapReduce distributed computing framework.
Questions tagged [mincemeat]
15 questions
13
votes
3 answers
global name 're' is not defined
I am new to python and working on a map reduce problem with mincemeat. I am getting the following error while running the mincemeat script.
$python mincemeat.py -p changeme localhost
error: uncaptured python exception, closing channel…
Satyajit Singh
- 191
- 1
- 2
- 6
4
votes
3 answers
How to pass a dictionary as value to a function in python
In python, I am using the mincemeat map-reduce framework
From my map function I would like to yield (k,v) in a loop, which would send the output to the reduce function (sample data given which is the output of my map function )
auth3 {'practical':…
user2339307
- 41
- 1
- 1
- 2
3
votes
1 answer
Mincemeat: supply additional parameters to map and reduce functions with closures
I would like to try out the Mincemeat map/reduce Python application for matrix multiplication. I am using Python 2.7. I found several web pages that describe how to do matrix multiplication using Hadoop in Java, and I have been referring to this one…
SuperSloMo
- 31
- 1
3
votes
1 answer
I am having difficulty using mincemeat in python for map-reduce to calculate wordcount of different files
Here is the code:
import glob
import mincemeat
import re
text_files = glob.glob('finalcount/1/*')
def file_contents(file_name):
f = open(file_name)
try:
return f.read()
finally:
f.close()
source = dict((file_name,…
amian
- 187
- 2
- 2
- 10
3
votes
1 answer
How do I distribute my task defined in example.py to two client computers using mincemeat?
I've downloaded the mincemeat.py with example from https://github.com/michaelfairley/mincemeatpy/zipball/v0.1.2
example.py as follows:
#!/usr/bin/env python
import mincemeat
data = ["Humpty Dumpty sat on a wall",
"Humpty Dumpty had…
priyank
- 148
- 3
- 16
2
votes
2 answers
Trouble setting up python mapreduce - mincemeat.py
I am trying to setup mincemeat.py on my system and am installing from source into a prefix folder. I use virtualenv and have python installed from source into a prefix directory. I am trying to install mincemeat.py to the same prefix directory.
I…
user1189851
- 4,861
- 15
- 47
- 69
2
votes
1 answer
Huge file as the data source for mincemeat.py
I am planning to use mincemeat.py for my map reduce task on a ~100GB file. After seeing the example code from mincemeat, it seems I need to input an in-memory dictionary as the data source. So, what is the right way to provide my huge file as the…
Karthikeyan
- 990
- 5
- 12
1
vote
1 answer
Best programming way to implement Map Reduce
We have a problem, which is an ideal case for applying MapReduce programming technique. The initial code for this is written in Python. Now we have the following options:
Use Hadoop and Java to implement the MapReduce part.
Use mincemeat and Python…
Saurabh Verma
- 6,328
- 12
- 52
- 84
1
vote
1 answer
Mincemeat.py - datasource division of files
I have created the following code:
#!/usr/bin/env python
import mincemeat
import glob
all_files = glob.glob('textfiles/*.txt')
def file_contents(file_name):
f = open(file_name)
try:
return f.read()
finally:
f.close()
#…
Anupam Bansal
- 17
- 3
1
vote
1 answer
Why do I keep getting this error in map-reduce while using mincemeat?
I just want to calculate word count from some 7500 files with some condition on which words to count. The program goes like this.
import glob
import mincemeat
text_files = glob.glob('../fldr/2/*')
def file_contents(file_name):
f =…
amian
- 187
- 2
- 2
- 10
1
vote
1 answer
"yield"ing a dictionary with keys as lists using mincemeat.py
I am trying to understand the map-reduce concept and looking at implementing small programs using mincemeat.py, an open source library for python.
I have obtained the simple word count for a bag of words using the mapper and reducer. However, I…
kausal_malladi
- 1,542
- 3
- 14
- 30
0
votes
1 answer
Error in running mincemeat
I'm getting an error while running mincemeat. The code runs ok in octo.py. I'm not putting the code as it is for a coursera assignment but would really prefer mincemeat. The error is as below:
C:\Python27>python mincemeat.py -p changeme…
anand
- 269
- 2
- 3
- 9
0
votes
1 answer
I cannot reach a dictionary in a file in the same directory
I try to write a homework about map-reduce. I run in a terminal:
ioannis@ioannis-desktop:~$ python hw3.py
then in another terminal:
ioannis@ioannis-desktop:~$ ls
a2.py la.py~ stopwords.py
active_output …
Ioannis Papaioannou
- 177
- 6
0
votes
1 answer
I cannot run mincemeat.py - a bunch of unknown stuff is printed to the terminal
I am trying to write a homework about map-reduce. I run in a terminal:
ioannis@ioannis-desktop:~$ python hw3.py
then in another terminal:
python mincemeat.py -p changeme localhost
Immediately in the former terminal, a bunch of stuff is…
Ioannis Papaioannou
- 177
- 6
0
votes
2 answers
Mincemeat map function returning dictionary
I am using a map reduce implementation called mincemeat.py. It contains a map function and reduce function. First off I will tell what I am trying to accomplish. I am doing a coursera course on bigdata where there is a programming assignment. The…
House
- 95
- 2
- 12