Proper Locking Method for Accessing Files from Multiple threads

Asked 2 years ago, Updated 2 years ago, 43 views

If you are accessing files from multiple threads in C language, please tell me which functions to lock.

We have created the following programs:

·Thread (1) is written to the file.
·Thread (2) is read against the file written by Thread (1).
·Threads (1) and (2) do not necessarily access the same file, but fopen() for each thread.
·Threads (1)(2) access the same file depending on the timing.

·(If the thread falls while locking itself, I would like to force the other thread side lock to continue processing (an additional error message will be printed in the log).

I was considering using the lock(fd, LOCK_EX) to unlock it, but
"The lock() locks for each file descriptor, so
Wouldn't it be possible to lock properly if fopen() was done separately?"
After being pointed out that , we are currently reviewing how to lock it.

linux c

2022-09-30 17:45

4 Answers

Linux may not be very useful, but

fopen_s() introduced in C11.Exclusive at the file system level, so you will not be able to reopen even if it is the same thread as just another process.

However, when will the preceding file users complete?There is a problem thatConsidering waiting for completion, exclusive synchronization such as pthread_mutex_lock proposed by user20098 is appropriate.

Threads (1)(2) do not access the same file, so mutex will have a slightly larger lock range.

Until you create mutex for each file.Will the file name/mutex pair be prepared in a structure and shared among threads?
If you do the right design like this

I don't want to fopen() each thread, preferably pass the file descriptors between threads.

I can't say that

Exclusive is a single process.

However, if writing and loading files is within a single process, I think it would be better to share the memory between threads in the process instead of going through the file.


2022-09-30 17:45

What Sayuri wanted to say in the answer to the previous question was when locking and releasing
I think the problem is that fp (or file descriptor) is different.

I think there is no problem with flock() as shown below.

#include<stdio.h>
# include <stdlib.h>
# include <unistd.h>
# include<sys/file.h>
# include <pthread.h>

#define N2

static FILE*fp0;

static void* test 0 (void*parm)
{
    intn=*(int*)parm;

    if(((fp0=fopen("test.txt", "a+"))==NULL){
        error("fopen");
        exit(1);
    }

    if(flock(fileno(fp0), LOCK_EX | LOCK_NB)!=0){
        error("block");
        exit(1);
    }

    return NULL;
}

static void*test1 (void*parm)
{
    intn=*(int*)parm;
    FILE*fp;

    if(flock(fileno(fp0), LOCK_UN | LOCK_NB)!=0){
        error("block");
        exit(1);
    }

    if(((fp=fopen("test.txt", "a+"))==NULL){
        error("fopen");
        exit(1);
    }

    if(flock(fileno(fp), LOCK_EX | LOCK_NB)!=0){
        error("block");
        exit(1);
    }

    fclose(fp);

    return NULL;
}

int main (void)
{
    pthread_tthr[N];
    pthread_attr_attr[N];
    int n [N];

    for (inti=0;i<N;i++)
        n[i] = i;

    pthread_attr_init(&attr[0]);
    pthread_create(&thr[0], &attr[0], test0, &n[0]);
    sleep(1);
    pthread_attr_init(&attr[1]);
    pthread_create(&thr[1], & attr[1], test1, & n[1]);

    for(inti=0;i<N;i++){
        void*r;
        pthread_join(thr[i], & r);
    }

    return 0;
}

You must pass fp0 to the test1() side thread that you did fopen() on the test0() side thread.
If you leave this fp0, there will be more and more FILE* that will not be fclose() if the thread drops many times.Therefore, fclose() will be required at the same time as the forced release, so it will need to be passed to the test1() side thread anyway, so I don't think this will be a problem.


2022-09-30 17:45

If it is an exclusive control of file reading and writing between threads in the same process, it would be easier to control it with pthread_mutex_lock().
(In other words, within the same process (PID), you will not be able to lock files between threads with fcntl().)

/*
 * Example code (no error determination)
 */
/* Exclusive control variable shared between threads */
pthread_mutex_tmutex=PTHREAD_MUTEX_INITIALIZE;

/*
 * Thread 1
 */
void*thread1(void*arg) {
    /*
     * abbreviation
     */
    pthread_mutex_lock (&mutex);
    FILE*fp=fopen("file", "r");
    /*
     * File Operation (Read)
     */
    fclose(fp);
    pthread_mutex_unlock (&mutex);
    /*
     * The following is an abbreviation.
     */
}
/*
 * Thread 2
 */
void*thread2(void*arg){
    /*
     * abbreviation
     */
    pthread_mutex_lock (&mutex);
    FILE*fp=fopen("file", "w+");
    /*
     * File Operation (Write)
     */
    fclose(fp);
    pthread_mutex_unlock (&mutex);
    /*
     * The following is an abbreviation.
     */
}

Also, if we take into account that the thread has ended abnormally, we will need another thread to monitor the status of the thread's life and death and exclusive variables.
I think it's a complicated process, so I think it's better to review the operating conditions.


2022-09-30 17:45

For your information, the following is a sample code using Open file description locks.

The difference from POSIX record locks is as follows:

The only difference is in fcntl command names:

F_OFD_SETLK installed of F_SETLK
F_OFD_SETLKW installed of F_SETLKW
F_OFD_GETLK installed of F_GETLK

Available only on Linux kernel 3.15 and later.

< fcntl(fd,F_SETLKW,...) and fcntl(fd,F_SETLK,...) which are POSIX record locks are commented out, but the contents of the output file (/tmp/output.dat) do not work as expected (top/code>).

#define_GNU_SOURCE
# include <stdio.h>
# include <stdlib.h>
# include <unistd.h>
# include <fcntl.h>
# include <pthread.h>

# define OUTPUT "/tmp/output.dat"

void*write_thread(void*id){
  long tid =(long)id;
  structure block lock = {
    .l_when=SEEK_SET,
    .l_start = 0,
    .l_len=0,//Lock whole file
  };

  int fd = open (OUTPUT, O_RDWR | O_CREAT, 0644);

  lock.l_type =F_WRLCK;
  fcntl(fd, F_OFD_SETLKW, & lock);
  // fcntl(fd, F_SETLKW, & lock);

  lseek(fd,0,SEEK_END);

  intlen; char buf [256];
  for(inti=0;i<5;i++){
    len=sprintf(buf, "tid=%ld, fd=%d, i=%d\n", tid, fd, i);
    write(fd, buf, len);
  }
  fsync(fd);

  lock.l_type =F_UNLCK;
  fcntl(fd, F_OFD_SETLK, & lock);
  // fcntl(fd, F_SETLK, & lock);
  close(fd);

  pthread_exit(NULL);
}

int main() {
  int num_threads=3;
  pthread_t threads [num_threads];

  truncate (OUTPUT, 0);
  for(inti=0;i<num_threads;i++){
    pthread_create(&threads[i], NULL, write_thread, (void*)i);
  }

  for(inti=0;i<num_threads;i++){
   pthread_join(threads[i], NULL);
  }

  return 0;
}


2022-09-30 17:45

If you have any answers or tips


© 2024 OneMinuteCode. All rights reserved.