Questions tagged [data-management]
312 questions
70
votes
10 answers
Read a CSV from github into R
I am trying to read a CSV from github into R:
latent.growth.data <- read.csv("https://github.com/aronlindberg/latent_growth_classes/blob/master/LGC_data.csv")
However, this gives me:
Error in file(file, "rt") : cannot open the connection
In…
histelheim
- 4,938
- 6
- 33
- 63
31
votes
9 answers
Quickly remove zero variance variables from a data.frame
I have a large data.frame that was generated by a process outside my control, which may or may not contain variables with zero variance (i.e. all the observations are the same). I would like to build a predictive model based on this data, and…
Zach
- 29,791
- 35
- 142
- 201
27
votes
3 answers
How to read a file backwards to find substring efficiently
I have a huge log file in this kind of structure:
"timestamp":…
Res ulma
- 273
- 2
- 7
25
votes
4 answers
NSCoding VS Core data
I've been searching for an article that explains NSCoding (NSKeyedArchiver...) advantages and disadvantages over use of CoreData (SQLite....).
There's a lot of options, I can implement my own custom binary reader/writer, or use plists/xml/json...…
LolaRun
- 5,526
- 6
- 33
- 45
24
votes
5 answers
How to create, structure, maintain and update data codebooks in R?
In the interest of replication I like to keep a codebook with meta data for each data frame. A data codebook is:
a written or computerized list that provides a clear and comprehensive description of the variables that will be included in the…
Fred
- 1,833
- 3
- 24
- 29
21
votes
3 answers
Realm migrations in Swift
I have a Realm Object modeled as so
class WorkoutSet: Object {
// Schema 0
dynamic var exerciseName: String = ""
dynamic var reps: Int = 0
// Schema 0 + 1
dynamic var setCount: Int = 0
}
I am trying to perform a…
Cody Weaver
- 4,756
- 11
- 33
- 51
10
votes
2 answers
Count data divided by year and by region in R
I have a very large (too big to open in Excel) biological dataset that looks something like this
year <- c(1990, 1980, 1985, 1980, 1990, 1990, 1980, 1985, 1985,1990,
1980, 1985, 1980, 1990, 1990, 1980, 1985, 1985,
…
colebrookson
- 831
- 7
- 18
9
votes
3 answers
reshaping k columns to 2 columns representing sequential pairs of the values of the k variables
I have a data frame like this:
id y1 y2 y3 y4
--+--+--+--+--
a |12|13|14|
b |12|18| |
c |13| | |
d |13|14|15|16
I want to reshape in such a way that I end with two columns. The above example would then become:
id from to
--+----+---
a…
Raul Torres
- 187
- 1
- 6
9
votes
4 answers
Mobile application data management
My question surrounds around one single point - data management in mobile application. I have created a mobile application where data comes from server. The data includes both text and images. Following are the steps I am doing for this :
First…
Nitish
- 13,845
- 28
- 135
- 263
7
votes
3 answers
Create a variable capturing the most frequent occurence by group
Define:
df1 <-data.frame(
id=c(rep(1,3),rep(2,3)),
v1=as.character(c("a","b","b",rep("c",3)))
)
s.t.
> df1
id v1
1 1 a
2 1 b
3 1 b
4 2 c
5 2 c
6 2 c
I want to create a third variable freq that contains the most frequent observation…
Fred
- 1,833
- 3
- 24
- 29
7
votes
3 answers
Android App Updating Without Losing Data
I'm new to Android development, so I'm trying to do an app that stores information about a warehouse. However, I'm afraid that if I perform an update, the user data will get lost. Do I have to manage which data should remain unchanged upon…
kickingnico
- 178
- 1
- 11
7
votes
3 answers
Simple Database Implementation for Educational Purpose
I would like to learn Database systems implementation in Depth. Is there an open-source simple implementation of database for educational purpose that I can go through the code? Like there are a lot of OS implementation (Minix, Pintos...). I am…
Alfred Zhong
- 6,773
- 11
- 47
- 59
6
votes
3 answers
Split a data frame into overlapping dataframes
I'm trying to write a function that behaves as follows, but it is proving very difficult:
DF <- data.frame(x = seq(1,10), y = rep(c('a','b','c','d','e'),2))
> DF
x y
1 1 a
2 2 b
3 3 c
4 4 d
5 5 e
6 6 a
7 7 b
8 8 c
9 9 d
10 10…
Zach
- 29,791
- 35
- 142
- 201
6
votes
1 answer
R: create a data frame out of a rolling window
Lets say I have a data frame with the following structure:
DF <- data.frame(x = 0:4, y = 5:9)
> DF
x y
1 0 5
2 1 6
3 2 7
4 3 8
5 4 9
what is the most efficient way to turn 'DF' into a data frame with the following structure:
w x y
1 0 5
1 1 6
2 1…
Zach
- 29,791
- 35
- 142
- 201
5
votes
4 answers
Select first 80 observations for each level in R
I have a data set that looks like this:
structure(list(A = structure(c(1L, 1L, 1L, 1L, 1L, 1L), .Label = c("1",
"2", "3", "4", "5", "6", "7", "8", "9", "10", "11", "12", "13",
"14", "15", "16", "17", "18", "19", "20", "21", "22", "23", "24",…
Kaye11
- 359
- 5
- 17