Java

Re: How to use JNI without setting Environment Variables

16-Oct-23 5:41

jschell wrote:
That is oddly phrased since they will need to have java installed somehow.

Why did you stopped there with the quote? As the next part of the phrase contained the reason why I'm not making the install Java:

Valentinor wrote:
but it comes in the app files with a license agreement, I'm using Eclipse Adoptium (...\MyApp\ThirdParty\Eclipse Adoptium\jre-17.0.7.7-hotspot).

So yeah, they don't need to install java, as it is included with the app.

jschell17-Oct-23 5:47

Re: How to use JNI without setting Environment Variables

17-Oct-23 5:47

Valentinor wrote:
So yeah, they don't need to install java,

On Windows a Java install consists basically of the following.
1. Lay down the files into the file system.
2. Create the registry data so the user can uninstall it.

So apparently what you are actually doing is giving the user the option to use the Java that is already installed or to use the VM that comes with your application.

Besides my other suggestion you might also want to consider whether the VM they have installed is the correct version for your appliction.

Valentinor17-Oct-23 19:26

Sending file though socket as byte[]

17-Oct-23 19:26

jschell wrote:
1. Lay down the files into the file system.

Yes or no, depending what you mean by "file system". When the app is installed, inside it's folder structure it will also have Eclipse Adoptium (...\MyApp\ThirdParty\Eclipse Adoptium\jre-17.0.7.7-hotspot)

jschell wrote:
2. Create the registry data so the user can uninstall it.

It will be for the app itself and not just for java, as Eclipse Adoptium will be a part of the app, integrated into it, and not separate. When the app is uninstalled, so will Eclipse Adoptium as it is part of its files. If the user has any other form of Java installed, they won't be affected in any way.

jschell wrote:
So apparently what you are actually doing is giving the user the option to use the Java that is already installed or to use the VM that comes with your application.

No. The app will only use Java that comes with it, as the app won't even look in another location if Java is installed, only inside it's file structure.

jschell wrote:
whether the VM they have installed is the correct version for your appliction

I explained in the above statement why this doesn't matter for my case.

jschell wrote:
As a suggestion only you might want to work on your error handling if it turns out Java isn't installed. Telling a user that a dll is missing is not going to help them nor you when they ask why it isn't working.

I never said that I don't have an error handling system already added. If any of Java files are missing or are corrupted, or any of the other app files are missing or are corrupted, an error will appear telling the user to run the check for corrupted files or reinstall the app, accompanied by an error message, that will tell me exactly which file is the problem, if it is missing or if it is corrupted.

JohnCodding25-Sep-23 20:52

25-Sep-23 20:52

I have a socket though which I'm sending files. If the file is under the buffer size (8192 in my case) then it is fine, but if the file is over that size, then when it writes to file, it writes the first chunk of 8192 bites every time. I'm using ObjectStream because beside the file, I'm sending other objects as well in the actual app, but this is the part of sending the file.

Client it sends the file data (testing):

Java

System.out.println("Starting client");
Socket socket = new Socket("localhost", port);
System.out.println("Client sending data");
ObjectOutputStream dataOut = new ObjectOutputStream(socket.getOutputStream());
BufferedInputStream is = new BufferedInputStream(new FileInputStream(inFile));
byte[] buffer = new byte[8192];
int sizeRead = 0;
while ((sizeRead = is.read(buffer, 0, 8192)) >= 0) {
    printByte(buffer, "in"); //Function that writes to a file as strings the byte[], and after 8192 you can see it is the first byte[] each time, also it writes at the end of file
    dataOut.writeObject(true);
    dataOut.writeObject(buffer);
    dataOut.writeObject(sizeRead);
    dataOut.flush();
}
is.close();
System.out.println("Done sending file");
dataOut.writeObject(false);
socket.close();
System.out.println("Client closed");

Server that creates the file:

Java

System.out.println("Starting server");
ServerSocket server = new ServerSocket(port);
System.out.println("Server waiting");
Socket socket = server.accept();
System.out.println("Client connected");
ObjectInputStream dataIn = new ObjectInputStream(socket.getInputStream());
BufferedOutputStream bos = new BufferedOutputStream(new FileOutputStream(outFile));
Object auxKeepReading = dataIn.readObject();
if (auxKeepReading instanceof Boolean) {
    while ((boolean) auxKeepReading) {
        Object auxBuffer = dataIn.readObject();
        if (!(auxBuffer instanceof byte[])) {
            System.out.println("Broke byte");
            break;
        }
        Object auxSize = dataIn.readObject();
        if (!(auxSize instanceof Integer)) {
            System.out.println("Broke int");
            break;
        }
        printByte((byte[]) auxBuffer, "out"); //Same as on client but different file, it writes at the end of file each time
        bos.write((byte[]) auxBuffer, 0, (int) auxSize);

        auxKeepReading = dataIn.readObject();
        if (!(auxKeepReading instanceof Boolean)) {
            System.out.println("Broke boolean");
            break;
        }
    }
} else {
    System.out.println("Broke boolean");
}
bos.close();
socket.close();
server.close();
System.out.println("Server stopped");

Why is it that it keeps sending the first chunk of byte[], and not what it reads new?

JohnCodding25-Sep-23 21:33

25-Sep-23 21:33

On client side after sending the size that it read, I added buffer = new byte[8192]; and now I don't have the problem and it is sending new data but there are 2 problems:
1. I don't know if this is what you should do;
2. The speed is limited to 3 MB/s when using IP instead of localhost, on an internet connection of 1.000 Mbps (~125 MB/s) upload/download (they are the same). I tried to increase the buffer size but still the same speed.

Richard MacCutchan25-Sep-23 22:32

Richard MacCutchan

25-Sep-23 22:32

I was looking at this code earlier and wondered why you sent the buffer size after the data. It makes much more sense to send the size first. Also there is no point in sending 8192 bytes if you have only read one byte from the input file. So only send the amount of data that has been read. Tuning TCP performance is not an exact science unfortunately, but you can find plenty of discussions of the subject on the internet.

JohnCodding25-Sep-23 23:44

25-Sep-23 23:44

Richard MacCutchan wrote:
I was looking at this code earlier and wondered why you sent the buffer size after the data. It makes much more sense to send the size first.

Because when they are used it doesn't matter the order they were sent, I didn't even think about that.

Richard MacCutchan wrote:
Also there is no point in sending 8192 bytes if you have only read one byte from the input file. So only send the amount of data that has been read.

I changed that now on client: dataOut.writeObject(Arrays.copyOf(buffer, sizeRead));

What about buffer = new byte[8192]; I added, is that how it should be done to send now data each time?

Java

dataOut.writeObject(true);
dataOut.writeObject(sizeRead);
dataOut.writeObject(Arrays.copyOf(buffer, sizeRead));
dataOut.flush();
buffer = new byte[8192];

Richard MacCutchan wrote:
Tuning TCP performance is not an exact science unfortunately, but you can find plenty of discussions of the subject on the internet.

From what I've seen they suggest two things most of the time:
1. Setting socket.setTcpNoDelay(true);. I tried that on both client and server bust still no use;
2. Using BufferedOutputStream/BufferedInputStream inside ObjectOutputStream/ObjectInputStream, but still the same 3 MB/s. Even after combining 1. and 2.

Richard MacCutchan25-Sep-23 23:58

Richard MacCutchan

25-Sep-23 23:58

JohnCodding wrote:
What about buffer = new byte[8192]; I added, is that how it should be done to send now data each time?

Yes, that is fine, as you allocate the buffer before you start the read/write loop. So each read from the input file will refill the same buffer. But I would be tempted to use ObjectOutputStream write(byte[](Java Platform SE 7 )[^], rather than writeObject, to send the byte data.

I have never had to do any TCP tuning so can't offer any useful suggestions, sorry.

JohnCodding26-Sep-23 0:53

26-Sep-23 0:53

Richard MacCutchan wrote:
But I would be tempted to use ObjectOutputStream write(byte[](Java Platform SE 7 )[^], rather than writeObject, to send the byte data.

In the past when I used socket but not for files, I had some problems when I was using combination of ObjectOutputStream writeBoolean(), writeInt(), writeUTF(), writeObject(), the order was correct when writing/reading but I was still getting some exceptions, and when I switch to only writeObject() I didn't had any problems. And with objects I have more control over what the server gets from the client when the data is wrong, and shadow ban the IP if it keeps sending the wrong data based on some checks. As java class files can easily be decompiled, they can recreate the code and send part of the data good then wrong one just to be jerks or for phishing data. I have some checks on the server to know when someone clearly wasn't supposed to send data, and ignore it, and it did help.

Richard MacCutchan wrote:
I have never had to do any TCP tuning so can't offer any useful suggestions, sorry.

Yeah, until now I didn't had problems, but now with files, also some that can be big, I do care about speed a lot. I'll keep looking and maybe I find a way to remove the limit. As there is clearly a limit because the speed stays around 3 MB/s and I did see someone complaining about the same exact speed, but there wasn't a solution on that thread.

jschell3-Oct-23 4:42

3-Oct-23 4:42

JohnCodding wrote:
The speed is limited to 3 MB/s when using IP instead of localhost, on an internet connection of 1.000 Mbps

Not quite sure what you are saying there but...

Yes localhost and the actual network are different. So a speed difference is expected. Matter of fact if it was the same I would expect (most likely) a problem with the actual measurement or that there was a hardware/software throttle happening in the client.

JohnCodding wrote:
internet connection ... upload/download (they are the same).

Noting also that I would not expect that. Even for a business. But I do want to emphasize that there is a difference between 'network' and 'internet'.

JohnCodding12-Oct-23 23:35

12-Oct-23 23:35

jschell wrote:
Noting also that I would not expect that. Even for a business.

SpeedTest[^]

jschell wrote:
Not quite sure what you are saying there but...

When transferring files over internet using the code I posted for sending/receiving, the transfer speed is always limited to 3 MB/s (bounces around 3 MB/s, it goes over or under with 100-200 KB/s) instead of the actual capabilities of internet speed and hardware (SSD).
If localhost is used, then the transfer is not limited to 3 MB/s.

jschell13-Oct-23 7:19

[Solved] Getting and setting a File's all attributes

13-Oct-23 7:19

JohnCodding wrote:
If localhost is used, then the transfer is not limited to 3 MB/s.

I meant that 'localhost' and 'network' should not have the same speed.

A connection, any connection, goes through actual hardware.

So your speedtest is connecting to a computer somewhere.

You client test app is connecting to a different computer somewhere.

So something is throttling it.

Best way is to test your app with another app that you control. You need two boxes. They can be local but you could also spin on a cloud box. Probably ideal on the cloud (internet connected.)

If that test shows a speed limitation then it is your apps problem (with I doubt.)
If not then it indicates that something at the other end is probably throttling it.

Also possible that your service provider is throttling it. They are 'faking' the results with speedtest by explicitly looking for it. Or you might be hitting a data limit with your app that the document (or don't.)

Valentinor24-Sep-23 22:47

Re: Getting and setting a File's all attributes

24-Sep-23 22:47

I made an app (Windows only) with which you can create a backup of a folder/file locally or to a server. I'm done with the copy/restore process, but when it comes to restoring I have a small technical problem. If the file in originally hidden, then this is easy as you can call Files.setAttribute(file.toPath(), "dos:hidden", true); which will make the file hidden on client side after it is created/restored, but there are files like desktop.ini which aren't simply hidden but are system protected hidden and the above code will only give it the hidden status, and not system protected hidden.

There is also Files.getAttribute() but as the name suggests it only returns the attribute value for a single attribute, and not all of them to easily be able to do Files.setAttributes(file.toPath(), Files.getAttributes(file.toPath()));. So my question is, is there a way to get a list will all the attributes a file has and then setting them in a more dynamically way and not having to call Files.getAttribute(); / Files.setAttribute(); for each of them line by line? If unfortunately there isn't one, and you do have to set them line by line, can someone give me a list with all the attributes for Windows OS like there is here for example with DateTimeFormatter[^] for pattern symbols?

modified 26-Sep-23 3:34am.

Victor Nijegorodov24-Sep-23 23:04

Victor Nijegorodov

24-Sep-23 23:04

I don't know how it was implemented in Java, however there is the API functions:
GetFileAttributesA function (fileapi.h) - Win32 apps | Microsoft Learn
SetFileAttributesA function (fileapi.h) - Win32 apps | Microsoft Learn
And here is the list of attributes:
File Attribute Constants (WinNT.h) - Win32 apps | Microsoft Learn

Re: Getting and setting a File's all attributes

Valentinor24-Sep-23 23:17

[Solution] Re: Getting and setting a File's all attributes

24-Sep-23 23:17

This doesn't look all that hard to do in C++, which makes me believe that there might be a way to do it in Java as well. If it was locally only (on the same machine) then I could make a dll and copy the attribues that way as both the backup and restored files are on the same machine, but the app also has the Socket part and sending the C++ attributes might give a a headache when trying to crate the code, compared to just sending java objects.

Even so, this is good info, thanks!

Valentinor25-Sep-23 6:26

Getting java.lang.RuntimeException: ZeroCode Step execution failed. Details:java.lang.NullP

25-Sep-23 6:26

I found a solution for the information when you open the Properties of a file, from the General tab, as for Details tab, that info it is copied with the file.

Example for General tab:

Java

DosFileAttributes dos = Files.readAttributes(original.toPath(), DosFileAttributes.class);
DosFileAttributeView dosView = Files.getFileAttributeView(copy.toPath(), DosFileAttributeView.class);
dosView.setArchive(dos.isArchive());
dosView.setHidden(dos.isHidden());
dosView.setReadOnly(dos.isReadOnly());
dosView.setSystem(dos.isSystem());
dosView.setTimes(dos.lastModifiedTime(), dos.lastAccessTime(), dos.creationTime());

modified 26-Sep-23 3:05am.

Member 160852744-Sep-23 5:46

Re: Getting java.lang.RuntimeException: ZeroCode Step execution failed. Details:java.lang.NullP

4-Sep-23 5:46

Hi,

My Junit test cases are compiled and running fine with

'zerocode-tdd', version: '1.3.1'
'rest-assured', version: '2.9.0'
'JAVA' version '1.8.0'
'gradle' version '6.9.2

Now it is upgraded to

'zerocode-tdd', version: '1.3.33'
'rest-assured', version: '5.3.0'
'JAVA' version '1.11.0'
'gradle' version '8.0.2

My test is compiled. while running getting below error

Test _mytest FAILED

java.lang.RuntimeException: ZeroCode Step execution failed. Details:java.lang.NullPointerException

snippet of code is myTest.java

@Test
@JsonTestCase("zerocode/11_Test.zero")
public void _11_Test() throws Exception {}

Could you please suggest what is wrong.
Thanks in Advance

Richard Deeming4-Sep-23 21:21

Richard Deeming

4-Sep-23 21:21

You've provided no details of the affected code, so we can't tell you precisely what the problem is.

All we can tell you is that a NullPointerException means that somewhere in your code you are trying to access a member of an object which has not been initialized, and is set to null.

You need to debug your code to find out what object is null when it shouldn't be. Then work out why it hasn't been initialized.

We can't do that for you.

"These people looked deep within my soul and assigned me a number based on the order in which I joined."
- Homer

Re: Getting java.lang.RuntimeException: ZeroCode Step execution failed. Details:java.lang.NullP

Member 160852744-Sep-23 23:55

Re: Getting java.lang.RuntimeException: ZeroCode Step execution failed. Details:java.lang.NullP

4-Sep-23 23:55

here is the file 11_Test.zero

{
"scenarioName": "Cleanup dr network",
"steps": [

// source1
{
"name": "step1_source1",
"url": "com.storvisor.backup.TestPreconditions",
"operation": "drNetworkCleanup",
"request": {
"____test_step": "1a_p001_cleanup_network_step1_source1",
"X-OperationID": "${RANDOM.NUMBER}", // actually System.currentTimeMillis()
"stdatasvcmgr": "http://10.198.74.45:9339/stdatasvcmgr",
"user": "[:]",
"rootsessionid": "564d1ec7-b0ec-e9d0-445f-7455b4d1bcdd-12532"
},
"assertions": {
"result": true
}
},

// source2
{
"name": "step1_source2",
"url": "com.storvisor.backup.TestPreconditions",
"operation": "drNetworkCleanup",
"request": {
"____test_step": "1a_p001_cleanup_network_step1_source2",
"X-OperationID": "${RANDOM.NUMBER}", // actually System.currentTimeMillis()
"stdatasvcmgr": "http://10.198.73.61:9339/stdatasvcmgr",
"user": "[:]",
"rootsessionid": "564dc67c-1f45-7270-6795-32a72f436514-23376"
},
"assertions": {
"result": true
}
},

// target
{
"name": "step1_target",
"url": "com.storvisor.backup.TestPreconditions",
"operation": "drNetworkCleanup",
"request": {
"____test_step": "1a_p001_cleanup_network_step1_target",
"X-OperationID": "${RANDOM.NUMBER}", // actually System.currentTimeMillis()
"stdatasvcmgr": "http://10.198.73.67:9339/stdatasvcmgr",
"user": "[:]",
"rootsessionid": "564dc295-844a-89ad-091e-6033987ce14b-29868"
},
"assertions": {
"result": true
}
}

]
}

under zerocode directory

File :myTest.java
------------------

package com.storvisor.backup;

import static org.junit.Assert.*;
import org.jsmart.zerocode.core.domain.JsonTestCase;
import org.jsmart.zerocode.core.domain.TargetEnv;
import org.jsmart.zerocode.core.runner.ZeroCodeUnitRunner;

import org.junit.FixMethodOrder;
import org.junit.Ignore;
import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.Timeout;
import org.junit.runner.RunWith;
import org.junit.runners.MethodSorters;

@TargetEnv("test.properties")
@FixMethodOrder(MethodSorters.NAME_ASCENDING)
@RunWith(PolicyTestUnitRunner.class)
public class myTest {

@Rule
public Timeout globalTimeout = Timeout.seconds(120);

@Test @JsonTestCase("zerocode/11_Test.zero")
public void _11_Test() throws Exception {}

}

---------------------------
PolicyTestUnitRunner.java
----------------------------
package com.storvisor.backup;

import org.jsmart.zerocode.core.runner.ZeroCodeUnitRunner;
import org.junit.runner.notification.RunNotifier;
import org.junit.runners.model.InitializationError;

public class PolicyTestUnitRunner extends ZeroCodeUnitRunner
{
public PolicyTestUnitRunner(Class klass) throws InitializationError
{
super(klass);
}

@Override
public void run(RunNotifier notifier)
{
PolicyTestListener failFastListener = new PolicyTestListener(notifier);
notifier.addListener(failFastListener);
super.run(notifier);
}
}

Thanks

Member 160852744-Sep-23 23:58

Re: Getting java.lang.RuntimeException: ZeroCode Step execution failed. Details:java.lang.NullP

4-Sep-23 23:58

Also Observed it is executing till source afterward it is throughing Java Runtime Exception
java.lang.RuntimeException: ZeroCode Step execution failed. Details:java.lang.NullPointerException

Member 160852745-Sep-23 0:09

Re: Getting java.lang.RuntimeException: ZeroCode Step execution failed. Details:java.lang.NullP

5-Sep-23 0:09

source1 is worked perfectly when it start consume soure2 it is througing

java.lang.RuntimeException: ZeroCode Step execution failed. Details:java.lang.NullPointerException.

Can you please let me know what is wrong on zero file
Thanking you

Richard MacCutchan5-Sep-23 0:21

Richard MacCutchan

5-Sep-23 0:21

No one here can tell you what is wrong with your file since we have no idea what the data represents. Also we have no idea where ther exception occurs, or why the variable in question has not been initialised. You are the only person that can diagnose the problem, by using the debugger, and examining what is happening when the code runs.

Re: Getting java.lang.RuntimeException: ZeroCode Step execution failed. Details:java.lang.NullP

jschell5-Sep-23 2:34

Re: Getting java.lang.RuntimeException: ZeroCode Step execution failed. Details:java.lang.NullP

5-Sep-23 2:34

Member 16085274 wrote:
Could you please suggest what is wrong.

The stack trace will tell you the exact line with the null pointer.

Member 160852745-Sep-23 5:11