Stack Overflow
Stack Overflow

Questions tagged [table-partitioning]

47 questions
3
votes
1 answer

How to get an element by id in a LIST partitioned table on postgres without scanning all tables for the index?

I have the following scenario in a pg major 11: DROP TABLE IF EXISTS public.agent_sessions_partitioned; CREATE TABLE public.agent_sessions_partitioned ( id uuid NOT NULL DEFAULT uuid_generate_v4(), …
Moondustt
  • 864
  • 1
  • 11
  • 30
2
votes
1 answer

SQL Server In-Memory table use case with huge data

I have SQL Server table with 160+ million records having continuous CRUD operations from UI, batch jobs etc. basically from multiple sources Currently I have partitioned the table on a column to have better performance on the table. I came across…
VR1256
  • 1,266
  • 4
  • 28
  • 56
2
votes
1 answer

How to perform insert overwrite dynamically on partitions of Delta file using PySpark?

I'm new to pyspark and looking for overwriting a delta partition dynamically. From the other resources available online I could see that spark supports dynamic partition by setting the below conf as…
Raji
  • 23
  • 1
  • 4
2
votes
2 answers

How to create a date partitioned table while using a loadjob in google Bigquery?

Would someone be able to explain how to create date partitioned table while using a loadjob in google Bigquery using JobConfig. https://cloud.google.com/bigquery/docs/creating-column-partitions#creating_a_partitioned_table_when_loading_data I…
Kishan Kumar
  • 173
  • 1
  • 13
1
vote
0 answers

How to choose between partitioned table vs partitioned view?

We are importing daily data (~600K rows) from Google GA4 into our data warehouse. We are trying to decide how we are going to store the data. We're considering either a partitioned table or a partitioned view that queries daily tables, with the…
Matthew Walk
  • 1,014
  • 2
  • 16
  • 36
1
vote
1 answer

How to find out the type of partitioning in a table in google bigquery using python apis

def partition(dataset1, dataset2): try: client.get_dataset(dataset2) print("Dataset {} already exists".format(dataset2)) except NotFound: print("Dataset {} not found".format(dataset2)) …
Max Daniel
  • 27
  • 2
1
vote
1 answer

I want to create a partition table month wise only 12 tables for multiple years

I am new to the partitioning of a table and I want to make a partition of the table by range type on the inserted_on column in this table the records are inserted around ~ 40000 daily I have tried creating a partition table as: CREATE TABLE…
Nitin
  • 11
  • 2
1
vote
1 answer

How to dynamically SELECT from manually partitioned table

Suppose I have table of tenants like so; CREATE TABLE tenants ( name varchar(50) ) And for each tenant, I have a corresponding table called {tenants.name}_entities, so for example for tenant_a I would have the following table. CREATE TABLE…
Florian Suess
  • 655
  • 6
  • 10
1
vote
1 answer

How to drop rows from partitioned hive table?

I need to drop specific rows from a Hive table, which is partitioned. These rows for deletion matches certain conditions, so entire partitions can not be dropped in order to do so. Lets say the table Table has three columns: partner, date and…
Dr Potato
  • 168
  • 1
  • 15
1
vote
1 answer

How do I change FILEGROUP of an existing Partition (SQL Server)

I have a very large table which I partitioned by months. I have also created filegroups for each year, so each filegroup would hold at most 12 partitions for that year. Here is the partition function: CREATE PARTITION FUNCTION…
Allen B
  • 13
  • 3
1
vote
0 answers

How to composite creation partitioning in rails by created_at and app_id using postgres

In project there is gonna to be more than 100 millions users and i want to partition it by range and list (composite) in other hand partition tables create by rails when our rows are more than 5 millions (any app id has 5 millions of user). and also…
uamin
  • 36
  • 3
1
vote
1 answer

Does MySQL Partitioning Require Special Queries To Take Advantage After Creating It?

I'm a newbie when it comes to MySQL/MariaDB partitions, and haven't created one yet, but am reading up on it. My first question is, if I partition a table by year and then month based on a dt_created DATETIME column, do I need to change the way I'm…
Volomike
  • 23,743
  • 21
  • 113
  • 209
1
vote
1 answer

Create aligned index on a foreign key column

I have a fact table which is partitioned along the PeriodDate column. CREATE TABLE MyFactTable ( PeriodDate DATE, OrderID INT CONSTRAINT MyFk FOREIGN KEY (OrderID) REFERENCES DimOrder(ID) ) I'd like create a partition aligned…
Avi
  • 1,066
  • 1
  • 15
  • 37
1
vote
0 answers

How to add empty file groups at lower and end of a partitioned table in SQL Server without dropping table, partition scheme and partition function?

I have a table called dbo.Audit. I initially had file groups like fgAudit2012, fgAudit2013 upto fgAudit2030 for years 2012 to 2030. The name of my partition function is pfMonthly and name of partition scheme is psMonthly. Each file groups (per year)…
Merin Nakarmi
  • 3,148
  • 3
  • 35
  • 42
0
votes
0 answers

Create Table fails because it is being used by active queries in this session

I am trying to implement dynamic partitioning on a table by using triggers on insert of data. Table: property(record_uuid uuid NOT NULL, change timestamp NOT NULL, value VARCHAR(50), PRIMARY KEY(property_id, record_uuid, change)) PARTITION BY…
Ankita Putatunda
  • 1
1
2 3 4