DYNAMICLY SECURED MEMORY AREA RELE
Asked 2 years ago, Updated 2 years ago, 104 viewspublic class Greeter{
public static String great (final String name) {
final String message = String.format("Hello, %s!",name);
return message;
}
public static void main(final String[]args) {
final String message=greet("world");
System.out.println(message);
// The area secured by String.format() will be recovered by GC.
}
}
I am thinking of replacing the greet method with the C library among the Java codes above.
I think the above code is legal for Java, but I think there is a problem with similar implementation of C because it leaks memory.
char*greet(char*name){
char*message=calloc(256,sizeof(char));
sprintf(message, "Hello, %s!", name);
return message;
}
int main(intargc, char**argv){
char*message=greet("world");
printf("%s\n", message);
return 0;
// The area reserved by calloc() is not released.
}
Therefore, I was thinking about how to release (and pair up) the dynamically secured areas as described below.
However, I don't think this kind of request is special in the first place, so I thought it might be generalized by idioms, patterns, library implementations, etc.
Are there any names or implementations that might be helpful (search keywords) for this issue?
The following are some of the responses I have been considering.
1.
Patterns that provide release.
Actually, I think there will be a mix of functions that are dynamically secured and functions that are not secured, so you may forget to call them.
void great_retval_dtor(char*retval){
free(retval);
}
// caller
char*message2=greet("destructor");
printf("%s\n", message2);
greet_retval_dtor(message2);
2.
The pattern in which the caller writes to the space reserved by the caller.
Not available unless the process is designed to meet the required size in advance.
(Or you need a separate function that tells you the size you need first.)
void greet_prepared_buffer(char*name, char*message){
sprintf(message, "Hello, %s!", name);
}
// caller
char*buff=calloc(256,sizeof(char));
greet_prepared_buffer("buffer", buffer);
printf("%s\n", buff);
free(buff);
3.
The pattern that callback returns to the caller.
It seems natural that the scope of the function matches the lifetime of the reserved area.
void great_callback(char*name, void(*cb)(char*message)){
char*message=calloc(256,sizeof(char));
sprintf(message, "Hello, %s!", name);
cb(message);
free(message);
}
// caller
void callback(char*message){
printf("%s\n", message);
}
greet_callback("callback", callback);
(Additional)
In fact, I'm thinking of implementing Java (Project Panama) as the caller and Rust as the caller C library.
However, I believe that this case does not depend on the implementation language (so I would like to know if there is a solution other than C language).
Below is the code that implements the previously mentioned pattern 2 (the pattern that the caller takes up space).
Caller Code (Java, 14-panama):
public static void main(final String[]args) {
final Scope scope=greeter_lib.scope();
final Pointer<Byte>name=scope.allocateCSstring("Rusty Tea");
final long size = 256;
final Pointer<Byte>message=scope.allocateArray(NativeTypes.UINT8,size).elementPointer();
greeter_lib.greet(name, message, size);
final String retval = Pointer.toString(message);
System.out.println(retval);
}
Calling Code (Rust, 1.41.0):
#[no_mangle]
pubunsafe external "C" fn greet(name:*const c_char, message:*mut c_char, count:size_t){
let name = CStr::from_ptr(name);
let name = name.to_str().unwrap();
let text=format!("Hello, {}!",name);
let text = CSstring::new(text).unwrap();
message.copy_from(text.as_ptr(), count);
}
5 Answers
The Java string is UTF-16, so if you want to interoperate, it is more efficient and easy to use a language that can handle UTF-16 on the native code side.The Java Native Interface (JNI) API function also provides the char*/const char* interface, which is a special form called Modified UTF-8 and costs extra conversion to UTF-16.
When interoperating native languages such as C/C++ with management languages such as Java/C#, native heap and management heap treat memory management differently.Whether it's a simple case where you receive and process managed and return managed, or whether you want to keep a pointer to a native heap on the management code side, the policy also changes.In addition, interoperability should follow the methods provided for each language and framework.Use JNI for Java, P/Invoke or COM interoperability for C#, or C++/CLI or C++/CX.
In the case of interaction between Java and native language, I think it would be better to use JNI honestly and implement the native side with C++ (C++11 or later).
(Project Panama probably doesn't have many users, so I don't think I can get proper development information.)
The jstring returned in JNIEnv::NewString() is a JNI local reference corresponding to Java's String type (manage heap). When native method call control is returned to Java, it is GC managed and release can be left to JVM.
//C++ side:
# include <jni.h>
# include <string>
external "C" JNIEXPORT jstring JNICALL
Java_com_example_Greeter_greet
(JNIEnv*env, jclass/*sender*/, jstring name) {
if(!name){
return nullptr;
}
const jsize srcLength=env->GetStringLength(name);
const jchar*nativeSrcBuffer=env->GetStringChars(name, nullptr);
jstring resultString=nullptr;
try{
std::u16 string str=u "Hello, ";
if(srcLength>0&&nativeSrcBuffer){
str+=std::u16 string(reinterpret_cast<const char16_t*>(nativeSrcBuffer), static_cast<std::size_t>(srcLength));
}
US>str+=u"!
resultString=env->NewString(reinterpret_cast<const jchar*>(str.c_str())), static_cast<jsize>(str.length()));
} catch(...){
}
if(nativeSrcBuffer){
env->ReleaseStringChars(name, nativeSrcBuffer);
nativeSrcBuffer=nullptr;
}
return resultString;
}
// Java Side:
package com.example;
public class Greater {
static {
System.loadLibrary("xxx");
}
public static native String great (String name);
...
}
However, Java/Native code interoperability should not be touched unless there is a good reason to accelerate compute-intensive operations or leverage the vast number of existing native code assets.In most cases, Java alone is preferred for code maintenance, and native code can degrade performance.
Generally speaking, native heap memory interactions between different languages or between different ABIs typically have buffer retention and release functions on the backend (library) side, which are secured and released on the frontend (application) side.In some cases, a separate buffer length query function is provided if the buffer size is unknown.The design of APIs written in the form of C language functions (e.g., OpenCL and Vulkan) should be helpful.
Higher-level language automates release processing to prevent memory leaks, so it is a rule to contain native resources in resource management frameworks such as RAII with constructor destructors for C++, AutoCloseable for Java, and Idisposable for C#.Many well-known C-language functional APIs have wrappers (bindings) for the corresponding C++/Java/C# developed either officially or by a third party, which should also be helpful.
In Windows APIs and COM (Component Object Model), GlobalAlloc()/GlobalFree() and CoTaskMemAlloc()/CoTaskMemFree() and SysAlocString()///OSSode> are provided by the APs.The ATL/MFC provides RAII wrappers.
You can use a pointer or intptr_t/uintptr_t in the C# code or IntPtr/UIntPtr in the C# code to hold the pointer or handle of the native heap, but you must use (bycode>).
Regarding the pattern in which the caller writes to the area reserved by the caller,
I have seen a pattern in which arguments receive buffer sizes and return values return the size they originally wanted.
size_tgreet(char*name, char*message, size_t message_size){
size_tneeded=9+strlen(name);
if(needed<=message_size){
sprintf(message, "hello, %s!", name);
}
returned needed;
}
int main (void)
{
size_t size=greet("buffer", NULL, 0);
char*buf=malloc(size);
greet("buffer", buf,size);
printf("%s\n", buf);
free(buf);
return 0;
}
Use the destructor as a class of c++
US>If c (assuming automatic variables) make it a macro using alloca()
I wonder if that's a realistic solution...
If you're talking about freeing up the memory you've saved in the heap, you'll find the words smart pointer and reference count in C++'s mind.I'm not familiar with Rust, but if you search for it, you'll find an official document, so why don't you refer to this area?
https://doc.rust-jp.rs/book/second-edition/ch15-00-smart-pointers.html
Basically, as you can guess, it is common and efficient to use prepared things without thinking.
It would be appropriate to have a heap in char*, string it, and use the standard library string instead of delete... in the destructor (as far as the document above shows, it also exists in Rust).
It seems that it is not impossible to implement in Pure C language.
https://stackoverflow.com/questions/799825/smart-pointers-safe-memory-management-for-c
Securing and liberating the C-language side of Java...I would avoid it.I think it's easier to manage and easier for users to use if you change it back to Java with JNI.
In many languages, such a feature is called Foreign Function Interface (FFI), so you will have to search per FFI+design,principle,concept by keyword.
As a direct answer to the question, opaque pointer was found as a specific idiom.
Other articles in the same direction as what I was wondering were:
In addition (and contrary to the original intent of the question, it becomes implementation language dependent).but),
The content of the article is quite close to the area in question, and I think the repository of the libraries described here is worth a closer look.
I have the impression that the topic of FFI at the moment is relatively often done to utilize existing program assets (there is no room to design the FFI API, and whether it can be adapted to the existing API will be of interest).
On the other hand, what Mozilla is trying to do with Rust is implementing new features using FFI for multi-platforms, so I felt that there was a lot of information that might be helpful to me.
(Here's proprietary study with little source.)
This time, there are two types of modules: "Caller" and "Caller".
When these two modules work together, I wonder if they can be classified into two types.
- The caller receives and processes content generated by the caller
- The caller receives and processes content generated by the caller
From here we will consider securing and destroying the memory space to store the content (hereafter referred to as a container for drawing reasons).
First of all, considering who is the right person to create the containers, it seems appropriate for the content producer to prepare the containers of the required size (hereinafter referred to as basic).
Content consumers generally don't know the size of the content, so it's hard to create the right container, and if it's too small, it's a bug (buffer overrun), and if it's too large, it's less efficient.
Then, why is the API in C language (e.g., scanf or strcpy) prepared by the e-container being the content consumer?
This may be due to a C language specific requirement to use the stack area as a container.
When I think about FFI, there are many languages that are not aware of the stack area (unconscious), so I don't think there is a strong demand for this.
However, even if you don't think about using the stack space, you may not want to leave the content producer to create the container.
In situations where you frequently ask the content producer to generate the content, you can use the previous one without having to create the container each time (the required container size is fixed and the current container can disappear).
In this case, you may want to create a container in advance and deliver the container first as a function of the content request to omit the container creation process.
(I think graphic processing is applicable, but I haven't been able to confirm it.)
As it will be done by performance requirements, there will be no situation in which this form naturally occurs, and it will be intentionally transformed from basic to container forwarding method.
Next, the right person to dispose of the container is that it is natural for content consumers to consume and do so when they no longer need it without much room to think about it, that is, content consumers do.
However, the implementation of the container disposal process and the implementation of the container creation process should be done by the same side, and calls may occur across API boundaries.
There was also a concern in the questionnaire that they forgot to destroy the container (leak).
As far as we've looked, this is true, but the language's equivalent (try-finally in Java and Scope's try-with-resources in ) and
Understanding Callback Methods.
"It is not necessarily necessary to avoid using it, but I had the impression that if I were to use the callback method instead of the ""em>basic"" mentioned above, I would need to have a proper intention (similar to the ""em>container forwarding method)."
Different languages and FFI library implementations have different considerations, and the actual level of difficulty seems higher than what comes to mind on your desk.
As far as we looked at, there were only two additional references to the callback method (outside the context of reuse of existing assets) that were linked at the beginning and the other total.
- YouTube:Nikita Baksalyar — All you need to know about the Rust FFI (Rust Hungary #2, 2017-10-24), from 28:15, summary is from 34
- Callbacks ar not ideal, but they work
In summary, basic is based, and container forwarding, callback should be modified from basic with clear intentions.
If you have any answers or tips
© 2024 OneMinuteCode. All rights reserved.