|
Unfortunately I'm new to "web area", but I still have to learn it.
Quote: You can check the existence of particular elements by reading the document surce.
So, I take it that the combination of DocumentBuilderFactory + DocumentBuilder + Document was the right choice, no?
Quote: Unless you tell us what the errors are it is difficult to make any useful suggestions.
Well this one is a little more complicated. It is different from site to site (my best guess is the way they are made).
When I get pass the 403 error, I'm getting 503 (couldn't find a way to fix this one), and there are some websites where I don't get either but the last error:
java.io.IOException: Server returned HTTP response code: 403 for URL:
java.io.IOException: Server returned HTTP response code: 503 for URL:
[Fatal Error] :1:3: The markup in the document preceding the root element must be well-formed.
Exception in thread "main" org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 3; The markup in the document preceding the root element must be well-formed.
All these errors point to this line:
Document doc = db.parse(new InputSource(httpURLConnection.getInputStream()));
|
|
|
|
|
All these errors are quite common and mean that the website cannot, or will not, provide you the web page that you request. Your code should first check the return status to see that you have received a 'good' page; see HTTP Status Codes — httpstatuses.com[^]. As I suggested before, you should first find a page that works, and look at the source to see exactly what gets returned to you. Remember, the data that is returned is controlled by the remote website and may not always be what you want, or think.
|
|
|
|
|
Unfortunately, for my idea to work, I was going to need for this to work with every website. And most likely those websites that have this type of "protection" are the ones that my users were going to use.
Well, thanks for you help!
|
|
|
|
|
That sounds very much like hacking, which is illegal, and something that no one here would help with.
|
|
|
|
|
Oh, far from it. I was going to add a function to my program, that was going to auto check if a new episode appeared, by searching to see if the "nextEpisode" button is present (here is where I was going to need this) on the page based on the link the user added as the last episode of show X. I think all websites have a next and previous episode button. This way, the user wouldn't had to open a webpage to see if a new episode aired or not, but just by simply looking at a column which would have had ✓ if a new episode is out, or an X if not.
|
|
|
|
|
Valentinor wrote: I was going to need for this to work with every website
Good luck with that.
I work for a company that does exactly that. And they have been attempting to fine tune that for 10 years.
|
|
|
|
|
Seems rather limited but...
- You must retrieve the site
- You must render the site. This can involve but is NOT limited to doing thing like loading other resources and running javascript.
- Process the results. This involves but is NOT limited to parsing the results.
- From the results search for the data that you need.
- Test, test, test, because that last step is going to be difficult.
|
|
|
|
|
Quote: Good luck with that.
I work for a company that does exactly that. And they have been attempting to fine tune that for 10 years.
Sure, then it will be no problem, after all I'm working alone on this project.
I'm not going to say that this part of the project will be forgotten, but I'll keep it as an idea, at the rate tech evolve each year, who knows, maybe an easier way will be added and I'll finally be able to add this feature to the project.
Quote: Seems rather limited but...
- You must retrieve the site
- You must render the site. This can involve but is NOT limited to doing thing like loading other resources and running javascript.
- Process the results. This involves but is NOT limited to parsing the results.
- From the results search for the data that you need.
- Test, test, test, because that last step is going to be difficult.
I get it what you say, and it "doesn't seem" like a lot of work, but I'll leave it for later, and focus on other features I want to add to the project for now. It would have been a really nice to have this too, but oh well, you can't have everything. Thanks!
|
|
|
|
|
I made a server to manage multiple clients. I create a new thread for each of them as you can see in my initModel(), and then all the threads goes in the run() method. The problem is that each client crashes after an undefined number of operation. The only error I get is this:
java[1133:134799] unrecognized type is 4294967295
02
java[1133:134799] *** Assertion failure in -[NSEvent _initWithCGEvent:eventRef:], /BuildRoot/Library/Caches/com.apple.xbs/Sources/AppKit/AppKit-1671.10.106/AppKit.subproj/NSEvent.m:1969
03
java[1133:134799] unrecognized type is 4294967295
04
java[1133:134799] *** Assertion failure in -[NSEvent _initWithCGEvent:eventRef:], /BuildRoot/Library/Caches/com.apple.xbs/Sources/AppKit/AppKit-1671.10.106/AppKit.subproj/NSEvent.m:1969
05
java[1133:134799] *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'Invalid parameter not satisfying: _type > 0 && _type <= kCGSLastEventType'
06
*** First throw call stack:
07
(
08
0 CoreFoundation 0x00007fff37e04e65 __exceptionPreprocess + 256
09
1 libobjc.A.dylib 0x00007fff63e5b720 objc_exception_throw + 48
10
2 CoreFoundation 0x00007fff37e1fab2 +[NSException raise:format:arguments:] + 98
11
3 Foundation 0x00007fff3a1c3d1d -[NSAssertionHandler handleFailureInMethod:object:file:lineNumber:description:] + 194
12
4 AppKit 0x00007fff352ac029 -[NSEvent _initWithCGEvent:eventRef:] + 3272
13
5 AppKit 0x00007fff355ff74c +[NSEvent eventWithCGEvent:] + 120
14
6 libglass.dylib 0x000000011c0086fb listenTouchEvents + 59
15
7 SkyLight 0x00007fff5df1141e _ZL19processEventTapDataPvjjjPhj + 148
16
8 SkyLight 0x00007fff5de0fc9e _XPostEventTapData + 278
17
9 SkyLight 0x00007fff5df1132c _ZL22eventTapMessageHandlerP12__CFMachPortPvlS1_ + 132
18
10 CoreFoundation 0x00007fff37d657cf __CFMachPortPerform + 282
19
11 CoreFoundation 0x00007fff37d656a9 __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE1_PERFORM_FUNCTION__ + 41
20
12 CoreFoundation 0x00007fff37d65607 __CFRunLoopDoSource1 + 527
21
13 CoreFoundation 0x00007fff37d4d689 __CFRunLoopRun + 2574
22
14 CoreFoundation 0x00007fff37d4ca28 CFRunLoopRunSpecific + 463
23
15 HIToolbox 0x00007fff36fe5b35 RunCurrentEventLoopInMode + 293
24
16 HIToolbox 0x00007fff36fe5774 ReceiveNextEventCommon + 371
25
17 HIToolbox 0x00007fff36fe55e8 _BlockUntilNextEventMatchingListInModeWithFilter + 64
26
18 AppKit 0x00007fff352a1eb7 _DPSNextEvent + 997
27
19 AppKit 0x00007fff352a0c56 -[NSApplication(NSEvent) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 1362
28
20 libglass.dylib 0x000000011bff646c +[GlassApplication enterNestedEventLoopWithEnv:] + 172
29
21 libglass.dylib 0x000000011bff6eaa Java_com_sun_glass_ui_mac_MacApplication__1enterNestedEventLoopImpl + 74
30
22 ??? 0x0000000106546667 0x0 + 4401161831
31
23 ??? 0x0000000106536040 0x0 + 4401094720
32
)
33
libc++abi.dylib: terminating with uncaught exception of type NSException
34
Java Result: 134
This is the code of my server:
public class ServerController {
002
003
@FXML
004
private TextArea textarea;
005
006
private StringBuilder contenutoTextArea = new StringBuilder("");
007
008
private ServerSocket s;
009
010
public void initModel() throws IOException {
011
contenutoTextArea.append("Waiting for connections\n");
012
textarea.setText(contenutoTextArea.toString());
013
s = new ServerSocket(5000);
014
new Thread() {
015
@Override
016
public void run() {
017
while (true) {
018
try {
019
new ThreadedEchoHandler(s);
020
} catch (IOException ex) {
021
Thread.currentThread().interrupt();
022
return;
023
}
024
}
025
}
026
}.start();
027
}
028
029
class ThreadedEchoHandler implements Runnable {
030
031
private Socket incoming;
032
033
private String nomeAccount = "";
034
035
ThreadedEchoHandler(ServerSocket serv) throws IOException {
036
incoming = serv.accept();
037
new Thread(this).start();
038
}
039
040
public void run() {
041
contenutoTextArea.append("Connected from: " + incoming.getLocalAddress() + "\n");
042
textarea.setText(contenutoTextArea.toString());
043
try {
044
045
try {
046
BufferedReader in = new BufferedReader(new InputStreamReader(incoming.getInputStream()));
047
nomeAccount = in.readLine();
048
} catch (IOException ex) {
049
System.out.println("Not works");
050
}
051
052
053
File dir = new File("src/server/" + nomeAccount);
054
String[] tmp = new String[100];
055
int i = 0;
056
for (File file : dir.listFiles()) {
057
if (file.isFile() && !(file.getName().equals(".DS_Store"))) {
058
try (BufferedReader br = new BufferedReader(new FileReader(file))) {
059
String line;
060
while ((line = br.readLine()) != null) {
061
tmp[i++] = line;
062
}
063
br.close();
064
} catch (IOException ex) {
065
System.out.println("Cannot read from file");
066
}
067
}
068
}
069
070
071
PrintWriter out = new PrintWriter(incoming.getOutputStream(), true);
072
for (int j = 0; j < i; j++) {
073
out.println(tmp[j]);
074
}
075
out.println("Fine");
076
077
} catch (IOException ex) {
078
System.out.println("Cannot send the strings to the client");
079
}
080
081
082
BufferedReader in;
083
String op;
084
boolean exit = false;
085
try {
086
in = new BufferedReader(new InputStreamReader(incoming.getInputStream()));
087
while (!exit && ((op = in.readLine()) != null)) {
088
if (op.equals("Fine")) {
089
exit = true;
090
contenutoTextArea.append("Connection closed.\n");
091
textarea.setText(contenutoTextArea.toString());
092
} else {
093
try {
094
clientChoise(in, op);
095
096
} catch (IOException ex) {
097
System.out.println("Wrong user input");
098
}
099
}
100
}
101
} catch (IOException ex) {
102
System.out.println("Unable to read messages");
103
} finally {
104
try {
105
incoming.close();
106
Thread.currentThread().interrupt();
107
return;
108
} catch (IOException ex) {
109
System.out.println("Cannot close the socket");
110
}
111
}
112
}
In particular, the communication between the server and the clients is focused on the PHASE 4, where the server waits for the inputs from the client. I've added the printStackTrace and on the Server I didn't get any error, but on the client each time I pressed a button or I did any operation I got these errors:
java[1133:134799] unrecognized type is 429496729502
2
java[1133:134799] *** Assertion failure in -[NSEvent _initWithCGEvent:eventRef:], /BuildRoot/Library/Caches/com.apple.xbs/Sources/AppKit/AppKit1671.10.106/AppKit.subproj/NSEvent.m:1969
And if I try to close the client manually I also get this one:
java.net.SocketException: Socket closed
02
at java.net.SocketInputStream.socketRead0(Native Method)
03
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
04
at java.net.SocketInputStream.read(SocketInputStream.java:171)
05
at java.net.SocketInputStream.read(SocketInputStream.java:141)
06
at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
07
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
08
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
09
at java.io.InputStreamReader.read(InputStreamReader.java:184)
10
at java.io.BufferedReader.fill(BufferedReader.java:161)
11
at java.io.BufferedReader.readLine(BufferedReader.java:324)
12
at java.io.BufferedReader.readLine(BufferedReader.java:389)
13
at mailbox.DataModel.askNumbOfEmail(DataModel.java:188)
14
at mailbox.ListController$2.run(ListController.java:65)
This is the code of the DataModel of the client, where all the operations and communications with the server are done:
public class DataModel {
002
003
private final String account = "alessandro@gmail.com";
004
005
private ObservableList<Email> emailList = FXCollections.observableArrayList(email
006
-> new Observable[]{email.IDProperty(), email.MittenteProperty()});
007
008
private final ObjectProperty<Email> currentEmail = new SimpleObjectProperty<>(null);
009
010
public ObjectProperty<Email> currentEmailProperty() {
011
return currentEmail;
012
}
013
014
public final Email getCurrentEmail() {
015
return currentEmailProperty().get();
016
}
017
018
public final String getAccountName() {
019
return account;
020
}
021
022
public final void setCurrentEmail(Email email) {
023
currentEmailProperty().set(email);
024
}
025
026
public ObservableList<Email> getEmailList() {
027
return emailList;
028
}
029
030
private Socket s;
031
032
public void connect() throws IOException {
033
try {
034
s = new Socket("127.0.0.1", 5000);
035
} catch (IOException io) {}
036
}
037
038
public Socket getSocket() {
039
return s;
040
}
041
042
private int numberOfEmail = 0;
043
044
public int getNumberOfEmail() {
045
return numberOfEmail;
046
}
047
048
public void loadData() throws IOException, ClassNotFoundException, ParseException {
049
050
try {
051
if (s == null || !isConnected()) {
052
connect();
053
}
054
ArrayList<Email> email = new ArrayList<Email>();
055
DateFormat format = new SimpleDateFormat("dd/MM/yyyy", new Locale("it", "IT"));
056
Date data;
057
058
059
try {
060
PrintWriter out = new PrintWriter(s.getOutputStream(), true);
061
out.println(account + "\n");
062
063
064
BufferedReader in = new BufferedReader(new InputStreamReader(s.getInputStream()));
065
String line;
066
String message[] = new String[5];
067
for (int j = 0; ((line = in.readLine()) != null) && (!line.equals("Fine"));)/> {
068
message[j++] = line;
069
if (j == 5) {
070
data = format.parse(message[3]);
071
email.add(new Email((Integer.parseInt(message[0])), message[1], account, message[2], message[4], data));
072
numberOfEmail++;
073
j = 0;
074
}
075
}
076
077
078
emailList = FXCollections.observableArrayList(email);
079
080
081
Collections.sort(emailList, (Email o1, Email o2) -> {
082
if (o1.getData() == null || o2.getData() == null) {
083
return 0;
084
}
085
return o1.getData().compareTo(o2.getData());
086
});
087
} finally {
088
System.out.println("Data Loaded");
089
}
090
} catch (SocketException se) {
091
se.printStackTrace();
092
emailList.setAll(null, null);
093
}
094
}
095
096
public void reLoadData() throws ParseException, IOException {
097
PrintWriter out = new PrintWriter(s.getOutputStream(), true);
098
out.println("Reload");
099
BufferedReader in = new BufferedReader(new InputStreamReader(s.getInputStream()));
100
String line;
101
String message[] = new String[5];
102
ArrayList<Email> email = new ArrayList<Email>();
103
DateFormat format = new SimpleDateFormat("dd/MM/yyyy");
104
Date data;
105
for (int j = 0; ((line = in.readLine()) != null) && (!line.equals("Fine"));)/> {
106
message[j++] = line;
107
if (j == 5) {
108
data = format.parse(message[3]);
109
email.add(new Email((Integer.parseInt(message[0])), message[1], account, message[2], message[4], data));
110
j = 0;
111
}
112
}
113
114
115
emailList = FXCollections.observableArrayList(email);
116
117
118
Collections.sort(emailList, (Email o1, Email o2) -> {
119
if (o1.getData() == null || o2.getData() == null) {
120
return 0;
121
}
122
return o1.getData().compareTo(o2.getData());
123
});
124
}
125
126
public void deleteMail(Email da_elim) throws IOException {
127
Socket p = getSocket();
128
int id_del = da_elim.getID();
129
emailList.remove(da_elim);
130
PrintWriter out = new PrintWriter(p.getOutputStream(), true);
131
out.println("Elimina");
132
out.println(Integer.toString(id_del));
133
}
134
135
public void closeConnection() throws IOException {
136
Socket p = getSocket();
137
PrintWriter out = new PrintWriter(p.getOutputStream(), true);
138
out.println("Fine");
139
p.close();
140
}
141
142
public void writeMail(String[] mail) throws IOException {
143
Socket p = getSocket();
144
PrintWriter out = new PrintWriter(p.getOutputStream(), true);
145
out.println("Scrivi");
146
for (int j = 0; j < mail.length; j++) {
147
out.println(mail[j]);
148
}
149
out.println("Stop");
150
151
}
152
153
public int askNumbOfEmail() throws IOException {
154
Socket p = getSocket();
155
PrintWriter out = new PrintWriter(p.getOutputStream(), true);
156
out.println("Numb");
157
BufferedReader in = new BufferedReader(new InputStreamReader(s.getInputStream()));
158
return Integer.parseInt(in.readLine());
159
160
}
161
162
public boolean isConnected() {
163
return getSocket()==null || getSocket().isConnected();
164
}
165
}
And this is the controller of my list view (of the client). Here I have a thread which checks continuously if there are new emails:
public class ListController {
02
03
@FXML
04
private ListView<Email> listView;
05
private DataModel model;
06
public int emailForClient;
07
08
public void initModel(DataModel model) throws IOException, ClassNotFoundException, ParseException {
09
if (this.model != null) {
10
throw new IllegalStateException("Model can only be initialized once");
11
}
12
this.model = model;
13
14
model.loadData();
15
16
listView.setItems(model.getEmailList());
17
18
listView.getSelectionModel().selectedItemProperty().addListener((obs, oldSelection, newSelection)
19
-> model.setCurrentEmail(newSelection));
20
model.currentEmailProperty().addListener((obs, oldEmail, newEmail) -> {
21
if (newEmail == null) {
22
listView.getSelectionModel().clearSelection();
23
} else {
24
listView.getSelectionModel().select(newEmail);
25
}
26
});
27
28
listView.setCellFactory(lv -> new ListCell<Email>() {
29
@Override
30
public void updateItem(Email mail, boolean empty) {
31
super.updateItem(mail, empty);
32
if (empty) {
33
setText(null);
34
} else if (mail != null) {
35
setText(mail.getMittente());
36
}
37
}
38
});
39
40
emailForClient = model.getNumberOfEmail();
41
new Thread() {
42
@Override
43
public void run() {
44
while (true) {
45
try {
46
int currentOnServer = model.askNumbOfEmail();
47
if (emailForClient != currentOnServer) {
48
model.reLoadData();
49
final int currentEmail = emailForClient;
50
Platform.runLater(() -> {
51
listView.setItems(model.getEmailList());
52
if (currentOnServer > currentEmail) {
53
new Alert(Alert.AlertType.INFORMATION, "You just received an email!").showAndWait();
54
}
55
});
56
emailForClient = currentOnServer;
57
}
58
} catch (IOException ex) {
59
ex.printStackTrace();
60
Thread.currentThread().interrupt();
61
return;
62
} catch (ParseException ex) {
63
ex.printStackTrace();
64
System.out.println("ParseException ERROR!");
65
}
66
}
67
}
68
}.start();
69
}
70
}
I've also noticed that if I leave the server runs, the fans of my computer start. What could cause that?
|
|
|
|
|
|
Member 14086702 wrote: Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason
In java threads terminate if an uncaught exception exits the thread.
Everything runs in a thread.
If all threads terminate then the application exits.
Solution is to catch exceptions and handle them. In a server/threaded application there are a number of strategies for how to do that.
Member 14086702 wrote: I've also noticed that if I leave the server runs, the fans of my computer start
Because the CPU and memory heats up as it is used and the fans cool them down
|
|
|
|
|
Hello,
I am studying this merge sort algorithm and a mystery still going on.
this merge sort algorithm :
class MergeSort
{
void merge(int arr[], int l, int m, int r)
{
int n1 = m - l + 1;
int n2 = r - m;
int L[] = new int [n1];
int R[] = new int [n2];
for (int i=0; i<n1; ++i)
L[i] = arr[l + i];
for (int j=0; j<n2; ++j)
R[j] = arr[m + 1+ j];
int i = 0, j = 0;
int k = l;
while (i < n1 && j < n2)
{
if (L[i] <= R[j])
{
arr[k] = L[i];
i++;
}
else
{
arr[k] = R[j];
j++;
}
k++;
}
while (i < n1)
{
arr[k] = L[i];
i++;
k++;
}
while (j < n2)
{
arr[k] = R[j];
j++;
k++;
}
}
void sort(int arr[], int l, int r)
{
if (l < r)
{
int m = (l+r)/2;
sort(arr, l, m);
sort(arr , m+1, r);
merge(arr, l, m, r);
}
}
static void printArray(int arr[])
{
int n = arr.length;
for (int i=0; i<n; ++i)
System.out.print(arr[i] + " ");
System.out.println();
}
public static void main(String args[])
{
int arr[] = {12, 11, 13, 5, 6, 7};
System.out.println("Given Array");
printArray(arr);
MergeSort ob = new MergeSort();
ob.sort(arr, 0, arr.length-1);
System.out.println("\nSorted array");
printArray(arr);
}
}
The problem is ubicating on the Main function that sorts arr[l..r] using merge() , here :
void sort(int arr[], int l, int r)
{
if (l < r)
{
int m = (l+r)/2;
sort(arr, l, m);
sort(arr , m+1, r);
merge(arr, l, m, r);
}
}
It's impossible to understand where is defined the value of "int l" and "int r" ?
Is it a question impossible to answer or will take time ?
Thanks anyway,
Best regard,
Intelego
|
|
|
|
|
Member 14084068 wrote: It's impossible to understand where is defined the value of "int l" and "int r" ?
They are the parameters to the sort method and are defined at the call:
ob.sort(arr, 0, arr.length-1);
|
|
|
|
|
Thanks,
of course, I was sure it was simple.
Just last question maybe easier :
In this part of the code :
void sort(int arr[], int l, int r)
{
if (l < r)
{
int m = (l+r)/2;
sort(arr, l, m);
sort(arr , m+1, r);
merge(arr, l, m, r);
I wonder if the result of (l+r)/2 is not an integer number (by example: 10,5), then is not it possible the machine become stuck?
I know I am wrong but what I ask is how what I said is made impossible inner the code?
in another way what the code have invented to prevent this?
Thanks very much,
Intelego
|
|
|
|
|
int m = (l+r)/2;
The variables m, l and r are all declared as integer types. So the result of any calculations with their values will always be integers. In the case of division, if there is a remainder then that will be discarded. For example the result of 7 divided by 2 will always be 3.
|
|
|
|
|
|
Quote: I am using BufferedReader for reading a response from the server.
while((String line = br.readLine()) != null)
System.out.println(line);
Above is my code. When I run I am getting only few lines but not all the lines.
If br completes reading a line then it should return null and come out of the loop. That is also not happening. It is just waiting for the line to read.
Any idea why this is happening ?
|
|
|
|
|
GiteHrudaya wrote: Any idea why this is happening ?
Because that is how networking works. Data is sent in packets which have nothing to do with "lines", it is just packets of bytes. You may get a complete message in a packet, but most of the time you will just get part of it, or part of some and all of another. So if you are trying to read a line at a time the receiver must wait until it receives the line end character(s). Until then it is in a wait condition.
|
|
|
|
|
GiteHrudaya wrote: If br completes reading a line then it should return null and come out of the loop.
Perhaps you misstated something you already know but that statement is not true.
It will exit the loop when the stream is done. Since it is not exiting the loop that means your stream is not done.
Or in other words you problem is not in that loop but rather how you are getting the stream in the first place.
I would suggest, unless you have a real reason otherwise that you should get the entire result first, probably buffered as a memory/byte stream, and then pass it to your line reading loop.
|
|
|
|
|
Please Guys i need some help for my Java GUI Project can u message me if u want to help this poor guy 
|
|
|
|
|
|
|
Hello,
I am trying to connect to mysql from Java with the following but getting this error
Exception in thread "main" Communications link failure
package testPackage;
import java.sql.Connection;
import java.sql.DriverManager;
import java.util.Properties;
public class Main {
public static void main(String[] args) throws Exception {
try {
Class.forName("com.mysql.jdbc.Driver");
Connection con=DriverManager.getConnection("jdbc:mysql://localhost:3306/student?useSSL=true","root","success");
System.out.print("Database is connected !");
con.close();
}
catch(Exception e)
{
System.out.println(e.getMessage());
throw e;
}
}
}
|
|
|
|
|
That message most likely means that JDBC cannot connect through port 3306, the mysql driver is not listening there, or some part of that path is not valid.
|
|
|
|
|
1
modified 9-Dec-18 15:29pm.
|
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
