DETAILED ACTION
Response to Arguments
The amendments filed 5/27/2026 have been entered and made of record.
Applicant's amendments and arguments filed 5/27/2026 have been considered but are moot in view of the new ground(s) of rejection because the Applicant has substantially amended at least independent claims, and Applicant's arguments in view of the amendments filed 5/27/2026 have been fully considered but they are not persuasive:
In the Applicant’s Remarks (on pages 2-3 of 6), regarding to amended claim limitation “detecting a plurality of contours of the point of reference after detecting the edges of the image”; Applicant also asserts that the cited references, particularly Sicconi (US 20200057487 A1) do not disclose above limitation;
However, the Examiner disagrees, because:
First, it is important to point out that the amendments to the independent claims including the limitations of previous dependent claims 2-7, however, these limitations are not connected to, or establish any association or contribution to the main solution/steps, which is to determine the pitch angle of a camera in a vehicle;
That is to say that the amendments including the limitations of previous dependent claims 2-7 are merely some well-known image processing and contour determining from the processed image; amended claim do not limit applying these results of blurring, detected edges, and/or determined contours in the determining the pitch angle of a camera in a vehicle;
That renders the claim interpretations into separated portions, rather than a unified flow of steps.
Second, since claim 1 does not define or limit the element of “a point of reference” in “positioning the camera of the vehicle to capture an image of a point of reference”; which can be interpreted to be any scenes, or contents of traffics that a vehicle might be presented, including roads, lanes, other vehicles, passengers, sidewalks, buildings, road blocks, parking sites, traffic lights…etc.,
So that Sicconi’s disclosures of captured image of the scenes, such as demonstrated in Fig. 14 (as reproduced below), Fig. 15, and Fig. 16, are suitable to be applied and referred as prior art into the discussions of the Office Action:
PNG
media_image1.png
578
888
media_image1.png
Greyscale
Sicconi discloses detecting a plurality of contours of the point of reference (see Sicconi: e.g., -- Due to the natural noise in the image, a simplification of the road region contour may be performed. Methods may include pre-processing the image with a Gaussian blur (5×5)--, in [0110];
Also see Sicconi: e.g., --further detection may be performed with the perspective shift. Lane detection methods may include, without limitation, color segmentation to pick out yellow and white lanes. Alternatively or additionally, a Canny threshold of the image may be taken, producing a detailed edge map of the scene. Canny edge map may capture more detail than necessary--, in [0114]; also see: --at step 1715 the motion detection analyzer 1324 determines a screen location on the digital screen 1308 of the rapid parameter change. In an embodiment, determining screen location may include identifying changing pixels according to a coordinate system as described above; identification may include, without limitation identifying coordinates of a boundary, geometric center, or the like of changed pixel area. Alternatively or additionally, determining screen location may include dividing the digital screen 1308 into a plurality of sections, regions of interest, and/or cells, for instance as described above, and identifying at least a cell and/or region of interest containing the rapid parameter change. A cell “contains” a rapid parameter change as used herein where the cell covers a portion of the digital screen 1308 where the rapid parameter change is occurring; in other words, a cell may “contain” the rapid parameter change where the cell contains at least a pixel undergoing the change. Identification may further include identifying a cell having a majority of changing pixels, a cell at a boundary and/or geometric center of a plurality of changing pixels, or the like.--, in [0128], and, -- Directional alerts may be generated to user based on detections of parameter changes and/or collision detection as described above. Collision detection routing may further isolate objects of motion in the image; these objects may then be passed through a classifier to be evaluated. Referring now to FIG. 23, a threshold method may classify objects from a heat map as described above. A threshold value is predefined, such as 165. A margin of error may be added. Further processing may be performed on the binarized blobs to filter certain aspect ratios and sizes. Bounding boxes of blobs may be generated and examined in the original image using a Haar Cascade, or any other suitable classifier. System may run on each individual frame using multi-threading to significantly improve response time. --, in [0142] {herein “segmentation” , and “identifying coordinates of a boundary, geometric center, or the like of changed pixel area.”, and/or “processing may be performed on the binarized blobs to filter certain aspect ratios and sizes. Bounding boxes of blobs may be generated and examined in the original image” read on claimed “contours”};
so that, Sicconi’s disclosures of “performing the road region contour” read on detecting a plurality of contours of the point of reference {of the scene}; and could be performed after the edge detection;
In the Applicant’s Remarks (on pages 2-3 of 6), Applicant states that “claim 1 as amended recites both (1) "detecting a plurality of edges of the image" and, as a separate subsequent step, (2) "detecting a plurality of contours of the point of reference after detecting the edges of the image." Edge detection and contour detection are distinct operations: edge detection identifies pixel-level intensity transitions across the image, whereas contour detection groups those detected edges into closed curves representing the boundaries of specific objects. Sicconi at para. [0114] performs only general edge detection of a road scene and does not perform extracting contours of a specific reference object from the edge-detected image”
And, further in amended claim 7, and claim 12 respectively, which includes the contents of: described in the specification at paragraph [0020], "the contours of the point of reference 22 (e.g., white rectangular plate) are detected" using, for example, "the FINDCONTOURS function of the OPENCV2 library." As-Filed Specification, paragraph [0020]. This is a targeted contour extraction operation performed on a specific reference object in a single captured image.
However, this limitation is still be rejected as a well-known image processing technology in vehicle images processing, further in view of newly found reference ZHANG (CN 115713736 A), because:
ZHANG discloses both (1) "detecting a plurality of edges of the image" and, as a separate subsequent step, (2) "detecting a plurality of contours of the point of reference after detecting the edges of the image.", and even detecting a plurality of contours based on "the FINDCONTOURS function of the OPENCV2 library." (see ZHANG: e.g., -- 1. Sobel operator extracting edge of image
after obtaining the picture of the driving in the road by the camera, firstly, the photo shot by the camera is
grey
wherein, respectively represent the red, green and blue RGB colour value in each pixel.
then extracting the vertical edge by Sobel operator. Order represents the grey information of an image,
wherein x and y represent the horizontal and vertical coordinate of the image pixel, the image is the partial
derivative of the horizontal direction and the vertical direction, and a pixel point of the image is The partial derivative in the angle direction can be respectively represented--, in page 8/14 of English version of ZHANG (CN 115713736 A), as provided as NPL with the Office Action; and,
-- if the angle equal to zero, namely representing the image has a longitudinal edge; otherwise, the image has a transverse edge; The edge information is extracted by edge density analysis based on line scanning.
1.2: Selected license plate
after performing binarization processing to the image, using cv2.findConfig in OpenCV, imutils.jin_contours function to search the contour of the detected object. We set a counter contours to count any object with a closed surface (below the python code):
searching the outline. Three input parameters: input image (binary image, black as background, white as target), contour search mode, contour approximation method. opencv2 returns two values: contours. The opencv3 returns to three values: img (image), countdown (contour), hierarchy (hierarchical structure)
Contours1=cv2.findContours(edged.copy(),cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE) – [10]; in pages 9-10/14 of English version of ZHANG (CN 115713736 A), as provided as NPL with the Office Action;
Also see: -- 2.1. The camera coordinate system of the camera
In order to systedescribe the coordinates of the object and image in the camera system, we define four coordinate systems: world coordinate system, camera coordinate system, image coordinate system and pixel coordinate system, the four coordinate systems are respectively appeared in each step of the camera imaging, the conversion relation between two two represents the conversion process in the imaging.
(1) world coordinate system: the three-dimensional coordinate system in the real world, for describing the specific position of any target in the reality physical space the origin of the coordinate system and the coordinate axis can be randomly selected, generally using represents;
(2) Camera coordinate system: observing the three-dimensional coordinate system of the reality physical space the angle of the camera, the origin of the coordinate system is selected as the imaging centre of the camera, the direction of the camera optical axis is the Z-axis of the coordinate system, generally using represents;
(3) Image coordinate system: defining a two-dimensional coordinate system on the imaging plane of the camera, for describing the target after camera imaging, the origin of the coordinate system is selected at the center of the imaging plane, generally using represents;--, in page 10/14 of English version of ZHANG (CN 115713736 A), as provided as NPL with the Office Action);
HANAWA (as modified by XU, HOLD and YU and Sicconi) and Zhang are combinable as they are in the same field of endeavor: processing images captured by vehicle camera . Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further modify HANAWA (as modified by XU, YU, and HOLD and Sicconi)’s method using Zhang’s teachings by including (1) "detecting a plurality of edges of the image" and, as a separate subsequent step, (2) "detecting a plurality of contours of the point of reference after detecting the edges of the image.", and even detecting a plurality of contours based on "the FINDCONTOURS function of the OPENCV2 library." to HANAWA (as modified by XU, YU and HOLD and Sicconi)’s image processing and performing contour detection and segmentations in order to search the contour of the detected object (see Zhang: e.g., in pages 8-10/14 of English version of ZHANG (CN 115713736 A), as provided as NPL with the Office Action).
Therefore, claims 1-12, 7-12, and 19-20 are still not patentably distinguishable over the prior art reference(s). Further discussions are addressed in the prior art rejection section below.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be /negated by the manner in which the invention was made.
Claims 1, 7-12, and 19-20 are rejected under 35 U.S.C. 103 as being patentable over HANAWA (JP 2018148520 A), in view of XU (WO 2023273376 A1), and further in view of HOLD (EP 2166510 A1), YU (CN 111508027 A), Sicconi (US 20200057487 A1), and further in view of ZHANG (CN 115713736 A).
Re Claim 1, HANAWA discloses a method for determining a pitch angle of a camera in a vehicle (see HANAWA: e.g., --and an attachment parameter calculation part 26 that calculates a pitch angle and a yaw angle of a camera 11 from the position in the imaging image of the elimination point. The elimination point is calculated in the imaging image from a moving locus of the tracking regions set in the imaging image, and calculates the pitch angle and the yaw angle of the camera 11 from the elimination point in the imaging image. Thus, the attachment direction parameter calculation device can avoids that the attachment parameter is calculate--, in abstract, and, Fig. 12, Fig. 13, and, in: -- as coordinates in the image space, the upper left is defined as the origin, the right direction is defined as the X axis, and the lower direction is defined as the Y axis, and the angle between the straight line parallel to the X axis and the vanishing line is defined as the camera roll angle (camera (Rotation angle with the optical axis as the axis). Also, using the focal length of the camera, the coordinates of the vanishing point, and the roll angle calculated above, the pitch angle of the camera (rotation angle about the direction parallel to the moving plane and perpendicular to the optical axis of the camera) is calculated. calculate. Further, the camera yaw angle (rotation angle about the direction perpendicular to the moving plane) is calculated using the focal length of the camera, the coordinates of the deep vanishing point, the vanishing line, and the calculated pitch angle.--, in pages 2-3 of English version of JP-2018148520-A as provided as NPL within this Office Action), comprising:
positioning the camera of the vehicle to capture an image of a point of reference, wherein the vehicle includes a vehicle body, and the camera is coupled to the vehicle body vehicle (see HANAWA: e.g., --and an attachment parameter calculation part 26 that calculates a pitch angle and a yaw angle of a camera 11 from the position in the imaging image of the elimination point. The elimination point is calculated in the imaging image from a moving locus of the tracking regions set in the imaging image, and calculates the pitch angle and the yaw angle of the camera 11 from the elimination point in the imaging image. Thus, the attachment direction parameter calculation device can avoids that the attachment parameter is calculate--, in abstract, Fig. 10, and, -- The image processing unit 12 detects the yaw angle of the agricultural vehicle 1 by processing the captured image acquired from the camera 11 and supplies the detected yaw angle to the ECU 13.--, in page 3 of English version of JP-2018148520-A as provided as NPL within this Office Action;
and, Fig. 12, Fig. 13, and, in: -- as coordinates in the image space, the upper left is defined as the origin, the right direction is defined as the X axis, and the lower direction is defined as the Y axis, and the angle between the straight line parallel to the X axis and the vanishing line is defined as the camera roll angle (camera (Rotation angle with the optical axis as the axis). Also, using the focal length of the camera, the coordinates of the vanishing point, and the roll angle calculated above, the pitch angle of the camera (rotation angle about the direction parallel to the moving plane and perpendicular to the optical axis of the camera) is calculated. calculate. Further, the camera yaw angle (rotation angle about the direction perpendicular to the moving plane) is calculated using the focal length of the camera, the coordinates of the deep vanishing point, the vanishing line, and the calculated pitch angle.--, in pages 2-3 of English version of JP-2018148520-A as provided as NPL within this Office Action;
Fig. 3, and, -- The target area setting unit 21 sets the target area 300 at the position of such an infinite point, and causes the storage unit 203 to store an image of a distant landscape in the target area 300 (hereinafter referred to as a target image). As shown in FIG. 3, the target area 300 is a rectangular area of a predetermined size
that is composed of a plurality of pixel values with the infinity point as the center. {herein “target area 300” aligns with claimed limitation of “a point of reference”}--, in pages 4-5 of English version of JP-2018148520-A as provided as NPL within this Office Action);
although HANAWA discloses determining the distance between the camera 11 and the target area of the image,
HANAWA however does not explicitly disclose that positioning the camera of the vehicle to capture an image of a point of reference while the vehicle remains stationary,
XU discloses positioning the camera of the vehicle to capture an image of a point of reference while the vehicle remains stationary (see XU: e.g., -- images can be acquired when the vehicle is stationary or running. When the vehicle is stationary, one image can be acquired.--, in pages 3-4 of English version of WO-2023273376-A1 as provided as NPL within this Office Action);
HANAWA and XU are combinable as they are in the same field of endeavor: vehicle pitch angle of camera and corresponding vehicle parameters calculations and calibrations. Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further modify HANAWA’s method using XU’s teachings by including positioning the camera of the vehicle to capture an image of a point of reference while the vehicle remains stationary, and capturing the image with the camera that includes the point of reference while the vehicle remains stationary to HANAWA’s positioning the camera of the vehicle and capturing the image in order to have a precise distance between the camera and the target (see Xu: e.g. in pages 3-4 of English version of WO-2023273376-A1 as provided as NPL within this Office Action),
HANAWA as modified by XU further disclose capturing the image with the camera of the vehicle that includes the point of reference while the vehicle remains stationary (see HANAWA: e.g., Fig. 10, and, -- The image processing unit 12 detects the yaw angle of the agricultural vehicle 1 by processing the captured image acquired from the camera 11 and supplies the detected yaw angle to the ECU 13.--, in page 3 of English version of JP-2018148520-A as provided as NPL within this Office Action;
and, Fig. 12, Fig. 13, and, in: -- as coordinates in the image space, the upper left is defined as the origin, the right direction is defined as the X axis, and the lower direction is defined as the Y axis, and the angle between the straight line parallel to the X axis and the vanishing line is defined as the camera roll angle (camera (Rotation angle with the optical axis as the axis). Also, using the focal length of the camera, the coordinates of the vanishing point, and the roll angle calculated above, the pitch angle of the camera (rotation angle about the direction parallel to the moving plane and perpendicular to the optical axis of the camera) is calculated. calculate. Further, the camera yaw angle (rotation angle about the direction perpendicular to the moving plane) is calculated using the focal length of the camera, the coordinates of the deep vanishing point, the vanishing line, and the calculated pitch angle.--, in pages 2-3 of English version of JP-2018148520-A as provided as NPL within this Office Action; and also see XU: e.g., -- images can be acquired when the vehicle is stationary or running. When the vehicle is stationary, one image can be acquired.--, in pages 3-4 of English version of WO-2023273376-A1 as provided as NPL within this Office Action);
importing the image from the camera into a controller, wherein the controller includes a processor and a non-transitory computer readable media in communication with the processor (see HANAWA: e.g., Fig. 10, and, -- The image processing unit 12 detects the yaw angle of the agricultural vehicle 1 by processing the captured image acquired from the camera 11 and supplies the detected yaw angle to the ECU 13.--, in page 3 of English version of JP-2018148520-A as provided as NPL within this Office Action;
and, Fig. 12, Fig. 13, and, in: -- as coordinates in the image space, the upper left is defined as the origin, the right direction is defined as the X axis, and the lower direction is defined as the Y axis, and the angle between the straight line parallel to the X axis and the vanishing line is defined as the camera roll angle (camera (Rotation angle with the optical axis as the axis). Also, using the focal length of the camera, the coordinates of the vanishing point, and the roll angle calculated above, the pitch angle of the camera (rotation angle about the direction parallel to the moving plane and perpendicular to the optical axis of the camera) is calculated. calculate. Further, the camera yaw angle (rotation angle about the direction perpendicular to the moving plane) is calculated using the focal length of the camera, the coordinates of the deep vanishing point, the vanishing line, and the calculated pitch angle.--, in pages 2-3 of English version of JP-2018148520-A as provided as NPL within this Office Action);
although HANAWA discloses determining the center point of “target area 300”, wherein the center of the point of reference in the image has a pixel position, and determining and the camera has a pixel density {herein “target area 300” aligns with claimed limitation of “a point of reference”} (see HANAWA: e.g., Fig. 3, and, -- The target area setting unit 21 sets the target area 300 at the position of such an infinite point, and causes the storage unit 203 to store an image of a distant landscape in the target area 300 (hereinafter referred to as a target image). As shown in FIG. 3, the target area 300 is a rectangular area of a predetermined size that is composed of a plurality of pixel values with the infinity point as the center. {herein “target area 300” aligns with claimed limitation of “a point of reference”}--, in pages 4-5 of English version of JP-2018148520-A as provided as NPL within this Office Action; and, --Here, a calculation method of the pitch angle of the camera 11 will be described with reference to FIGS. 9 and 10. As shown in FIG. 9, the coordinates on the captured image are set with the I coordinate in the right direction and the J coordinate in the upward direction with the lower left corner as the origin, and the unit is a pixel (for example, the size of the image in FIG. 9 is It is 752 pixels wide and 480 pixels long). Further, as shown in FIG. 10, the coordinates in the real space are the X axis in the horizontal direction, the Y axis in the height direction, and the Z axis in the straight traveling direction of the agricultural vehicle 1 with the position on the ground directly below the camera 11 as the origin. The axis shall be set.
The pitch angle of the camera 11 is θvc shown in FIG. 10A, and is the angle formed by the optical axis direction of the camera 11, that is, the screen center direction Jc, and the direction parallel to the ground, that is, the horizon direction Jv. . On the captured image as shown in FIG. 9, Jv is the J coordinate value in the horizon direction, Jc is the J coordinate of the screen center, and the pitch angle θvc is calculated by the following equation (P1).
tan (θvc) = PWH × (Jv−Jc) (P1)
Note that PWH is a tan value of the viewing angle per pixel.--, in pages 7-8 of English version of JP-2018148520-A as provided as NPL within this Office Action);
HANAWA as modified by XU however do not explicitly disclose determining a center of the point of reference in the image;
HOLD disclose determining a center of the point of reference in the image (see HOLD: e.g., -- the position of the characteristic pixels in the camera image is determined by means of a modified radial symmetry transformation. The characteristic pixels include in the camera image usually a range of several pixels, if the resolution of the camera is large enough. By means of the modified radial symmetry transformation, for example, the centers of circular points in the camera image can be determined with subpixel accuracy… In order to completely determine the position and the orientation of the camera arranged in the vehicle, a yaw angle of the camera with respect to the vehicle coordinate system is preferably additionally determined.--, in pages 3-4 of English version of EP 2166510 A1 as provided as NPL within this Office Action);
HANAWA (as modified by XU) and HOLD are combinable as they are in the same field of endeavor: vehicle pitch angle of camera and corresponding vehicle parameters calculations and calibrations. Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further modify HANAWA (as modified by XU)’s method using HOLD’s teachings by including determining a center of the point of reference in the image to HANAWA (as modified by XU)’s determining the center of the target region in order to determine the position of the characteristic pixels in the camera image (see HOLD: e.g. in pages 3-4 of English version of EP 2166510 A1 as provided as NPL within this Office Action),
HANAWA as modified by XU and HOLD however still do not disclose transforming the image from a Blue- Green-Red (BGR) color space into a gray color space after importing the image from the camera into the controller,
Yu discloses transforming the image from a Blue- Green-Red (BGR) color space into a gray color space after importing the image from the camera into the controller (see YU: e.g., -- the camera shoots the first image at the first position, and can be sent to the terminal device, and the camera shoots the second image at the second position, and also can be sent to the terminal device. The first image and the second image are typically RGB (Red Green Blue) images, so that the first image and the second image can be converted to a grey scale by the formula: Y=R*0.299 + G* 0.587 + B*0.114, i.e., the grey scale processing is performed. then performing distortion correction processing to the grey image corresponding to the first image, obtaining the first ideal image, and performing distortion correction processing to the grey image corresponding to the second image to obtain the second ideal image.--, in 6-7 para. of page 10 of English version of CN 111508027 A as provided as NPL within this Office Action);
HANAWA (as modified by XU and HOLD) and YU are combinable as they are in the same field of endeavor: vehicle pitch angle of camera and corresponding vehicle parameters calculations and calibrations. Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further modify HANAWA (as modified by XU and HOLD)’s method using YU’s teachings by including transforming the image from a Blue- Green-Red (BGR) color space into a gray color space after importing the image from the camera into the controller to HANAWA (as modified by XU and HOLD)’s image processing in order to perform grey scale processing (see YU: e.g., in 6-7 para. of page 10 of English version of CN 111508027 A as provided as NPL within this Office Action);
HANAWA as modified by XU, HOLD and YU however still do not disclose blurring the image after transforming the image from the BGR color space into the gray color space;
Sicconi discloses blurring the image after transforming the image from the BGR color space into the gray color space (see Sicconi: e.g., -- Due to the natural noise in the image, a simplification of the road region contour may be performed. Methods may include pre-processing the image with a Gaussian blur (5×5)--, in [0110]);
HANAWA (as modified by XU, HOLD and YU) and Sicconi are combinable as they are in the same field of endeavor: processing images captured by vehicle camera . Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further modify HANAWA (as modified by XU, YU, and HOLD)’s method using Sicconi’s teachings by including blurring the image after transforming the image from the BGR color space into the gray color space to HANAWA (as modified by XU, YU and HOLD)’s image processing in order to reduce the image noises (see Sicconi: e.g., in [0110]);
HANAWA as modified by XU, HOLD, YU and Sicconi further disclose wherein a Gaussian blur is used to blur the image (see Sicconi: e.g., in [0110]);
detecting a plurality of edges of the image after blurring the image, wherein a Canny edge detector is used to detect the plurality of edges of the image (see Sicconi: e.g., --further detection may be performed with the perspective shift. Lane detection methods may include, without limitation, color segmentation to pick out yellow and white lanes. Alternatively or additionally, a Canny threshold of the image may be taken, producing a detailed edge map of the scene. Canny edge map may capture more detail than necessary--, in [0114]);
HANAWA as modified by XU, HOLD, YU and Sicconi further disclose discloses detecting a plurality of contours of the point of reference (see Sicconi: e.g., -- Due to the natural noise in the image, a simplification of the road region contour may be performed. Methods may include pre-processing the image with a Gaussian blur (5×5)--, in [0110];
Also see Sicconi: e.g., --further detection may be performed with the perspective shift. Lane detection methods may include, without limitation, color segmentation to pick out yellow and white lanes. Alternatively or additionally, a Canny threshold of the image may be taken, producing a detailed edge map of the scene. Canny edge map may capture more detail than necessary--, in [0114]; also see: --at step 1715 the motion detection analyzer 1324 determines a screen location on the digital screen 1308 of the rapid parameter change. In an embodiment, determining screen location may include identifying changing pixels according to a coordinate system as described above; identification may include, without limitation identifying coordinates of a boundary, geometric center, or the like of changed pixel area. Alternatively or additionally, determining screen location may include dividing the digital screen 1308 into a plurality of sections, regions of interest, and/or cells, for instance as described above, and identifying at least a cell and/or region of interest containing the rapid parameter change. A cell “contains” a rapid parameter change as used herein where the cell covers a portion of the digital screen 1308 where the rapid parameter change is occurring; in other words, a cell may “contain” the rapid parameter change where the cell contains at least a pixel undergoing the change. Identification may further include identifying a cell having a majority of changing pixels, a cell at a boundary and/or geometric center of a plurality of changing pixels, or the like.--, in [0128], and, -- Directional alerts may be generated to user based on detections of parameter changes and/or collision detection as described above. Collision detection routing may further isolate objects of motion in the image; these objects may then be passed through a classifier to be evaluated. Referring now to FIG. 23, a threshold method may classify objects from a heat map as described above. A threshold value is predefined, such as 165. A margin of error may be added. Further processing may be performed on the binarized blobs to filter certain aspect ratios and sizes. Bounding boxes of blobs may be generated and examined in the original image using a Haar Cascade, or any other suitable classifier. System may run on each individual frame using multi-threading to significantly improve response time. --, in [0142] {herein “segmentation” , and “identifying coordinates of a boundary, geometric center, or the like of changed pixel area.”, and/or “processing may be performed on the binarized blobs to filter certain aspect ratios and sizes. Bounding boxes of blobs may be generated and examined in the original image” read on claimed “contours”};
so that, Sicconi’s disclosures of “performing the road region contour” read on detecting a plurality of contours of the point of reference {of the scene}; and could be performed after the edge detection);
HANAWA as modified by XU, HOLD, YU and Sicconi however still do not explicitly disclose that (2) "detecting a plurality of contours of the point of reference after detecting the edges of the image.", and even detecting a plurality of contours based on "the FINDCONTOURS function of the OPENCV2 library.";
ZHANG discloses both (1) "detecting a plurality of edges of the image" and, as a separate subsequent step, (2) "detecting a plurality of contours of the point of reference after detecting the edges of the image.", and even detecting a plurality of contours based on "the FINDCONTOURS function of the OPENCV2 library." (see ZHANG: e.g., -- 1. Sobel operator extracting edge of image
after obtaining the picture of the driving in the road by the camera, firstly, the photo shot by the camera is
grey
wherein, respectively represent the red, green and blue RGB colour value in each pixel.
then extracting the vertical edge by Sobel operator. Order represents the grey information of an image,
wherein x and y represent the horizontal and vertical coordinate of the image pixel, the image is the partial
derivative of the horizontal direction and the vertical direction, and a pixel point of the image is The partial derivative in the angle direction can be respectively represented--, in page 8/14 of English version of ZHANG (CN 115713736 A), as provided as NPL with the Office Action; and,
-- if the angle equal to zero, namely representing the image has a longitudinal edge; otherwise, the image has a transverse edge; The edge information is extracted by edge density analysis based on line scanning.
1.2: Selected license plate
after performing binarization processing to the image, using cv2.findConfig in OpenCV, imutils.jin_contours function to search the contour of the detected object. We set a counter contours to count any object with a closed surface (below the python code):
searching the outline. Three input parameters: input image (binary image, black as background, white as target), contour search mode, contour approximation method. opencv2 returns two values: contours. The opencv3 returns to three values: img (image), countdown (contour), hierarchy (hierarchical structure)
Contours1=cv2.findContours(edged.copy(),cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE) – [10]; in pages 9-10/14 of English version of ZHANG (CN 115713736 A), as provided as NPL with the Office Action;
Also see: -- 2.1. The camera coordinate system of the camera
In order to systedescribe the coordinates of the object and image in the camera system, we define four coordinate systems: world coordinate system, camera coordinate system, image coordinate system and pixel coordinate system, the four coordinate systems are respectively appeared in each step of the camera imaging, the conversion relation between two two represents the conversion process in the imaging.
(1) world coordinate system: the three-dimensional coordinate system in the real world, for describing the specific position of any target in the reality physical space the origin of the coordinate system and the coordinate axis can be randomly selected, generally using represents;
(2) Camera coordinate system: observing the three-dimensional coordinate system of the reality physical space the angle of the camera, the origin of the coordinate system is selected as the imaging centre of the camera, the direction of the camera optical axis is the Z-axis of the coordinate system, generally using represents;
(3) Image coordinate system: defining a two-dimensional coordinate system on the imaging plane of the camera, for describing the target after camera imaging, the origin of the coordinate system is selected at the center of the imaging plane, generally using represents;--, in page 10/14 of English version of ZHANG (CN 115713736 A), as provided as NPL with the Office Action);
HANAWA (as modified by XU, HOLD and YU and Sicconi) and Zhang are combinable as they are in the same field of endeavor: processing images captured by vehicle camera . Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further modify HANAWA (as modified by XU, YU, and HOLD and Sicconi)’s method using Zhang’s teachings by including (1) "detecting a plurality of edges of the image" and, as a separate subsequent step, (2) "detecting a plurality of contours of the point of reference after detecting the edges of the image.", and even detecting a plurality of contours based on "the FINDCONTOURS function of the OPENCV2 library." to HANAWA (as modified by XU, YU and HOLD and Sicconi)’s image processing and performing contour detection and segmentations in order to search the contour of the detected object (see Zhang: e.g., in pages 8-10/14 of English version of ZHANG (CN 115713736 A), as provided as NPL with the Office Action);
HANAWA as modified by XU, HOLD, and YU and Sicconi and Zhang further disclose determining a determined pitch angle using the pixel position of the center of the point of reference and the pixel density of the camera (see HANAWA: e.g., Fig. 3, and, -- The target area setting unit 21 sets the target area 300 at the position of such an infinite point, and causes the storage unit 203 to store an image of a distant landscape in the target area 300 (hereinafter referred to as a target image). As shown in FIG. 3, the target area 300 is a rectangular area of a predetermined size that is composed of a plurality of pixel values with the infinity point as the center. {herein “target area 300” aligns with claimed limitation of “a point of reference”}--, in pages 4-5 of English version of JP-2018148520-A as provided as NPL within this Office Action; and, --Here, a calculation method of the pitch angle of the camera 11 will be described with reference to FIGS. 9 and 10. As shown in FIG. 9, the coordinates on the captured image are set with the I coordinate in the right direction and the J coordinate in the upward direction with the lower left corner as the origin, and the unit is a pixel (for example, the size of the image in FIG. 9 is It is 752 pixels wide and 480 pixels long). Further, as shown in FIG. 10, the coordinates in the real space are the X axis in the horizontal direction, the Y axis in the height direction, and the Z axis in the straight traveling direction of the agricultural vehicle 1 with the position on the ground directly below the camera 11 as the origin. The axis shall be set.
The pitch angle of the camera 11 is θvc shown in FIG. 10A, and is the angle formed by the optical axis direction of the camera 11, that is, the screen center direction Jc, and the direction parallel to the ground, that is, the horizon direction Jv. . On the captured image as shown in FIG. 9, Jv is the J coordinate value in the horizon direction, Jc is the J coordinate of the screen center, and the pitch angle θvc is calculated by the following equation (P1).
tan (θvc) = PWH × (Jv−Jc) (P1)
Note that PWH is a tan value of the viewing angle per pixel.--, in pages 7-8 of English version of JP-2018148520-A as provided as NPL within this Office Action; also see HOLD: e.g., --by arranging a camera at a first distance relative to a geometric pattern such that at least three characteristic pattern points not lying on a straight line are located in a viewing area of the camera , where the distances between the characteristic points are known, and the camera takes a first image of the geometric pattern. A height, a roll angle and a pitch angle of the camera with respect to a coordinate system defined by the vehicle, here also as Vehicle coordinate system is determined from the location of at least three characteristic pixels in the image taken by the camera that correspond to the characteristic pattern points, by optimizing an imaging model that describes the imaging characteristics of the camera.--, in page 2 of English version of EP 2166510 A1 as provided as NPL within this Office Action);
determining whether the determined pitch angle of the camera is equal to a predetermined pitch angle (see HANAWA: e.g., --the position of the horizon based on the yaw angle and pitch angle of the camera 11 set in advance, and sets the target area at the calculated position.--, in page 4 of English version of JP-2018148520-A as provided as NPL within this Office Action;
see HOLD: e.g., -- Once the position and orientation of the camera located in the vehicle has been determined, the height, roll, pitch and yaw angles of the camera relative to the vehicle coordinate system may be passed as parameters to an image evaluation algorithm of a driver assistance system…. The calibration unit is further configured to perform an optimization of the imaging model describing the imaging characteristics of the camera. By optimizing the imaging model, the height, roll angle and pitch angle of the camera are determined by means of the location of characteristic pixels in an image captured by the camera. --; in pages 3-4 of English version of EP 2166510 A1 as provided as NPL within this Office Action); and
in response to determining that the determined pitch angle of the camera is not equal to the predetermined pitch angle, rotating the camera relative to the vehicle body until the determined pitch angle is equal to the predetermined pitch angle (see: HANAWA: e.g., Fig. 13, and, --That is, as coordinates in the image space, the upper left is defined as the origin, the right direction is defined as the X axis, and the lower direction is defined as the Y axis, and the angle between the straight line parallel to the X axis and the vanishing line is defined as the camera roll angle (camera (Rotation angle with the optical axis as the axis). Also, using the focal length of the camera, the coordinates of the vanishing point, and the roll angle calculated above, the pitch angle of the camera (rotation angle about the direction parallel to the moving plane and perpendicular to the optical axis of the camera) is calculated. calculate. Further, the camera yaw angle (rotation angle about the direction perpendicular to the moving plane) is calculated using the focal length of the camera, the coordinates of the deep vanishing point, the vanishing line, and the calculated pitch angle.--, in pages 2-3 of English version of JP-2018148520-A as provided as NPL within this Office Action;
also see HOLD: e.g., -- The camera 13 has an optical axis and an image plane defining image sensor, e.g. a CCD sensor, wherein the intersection of the optical axis with the imaging plane is used as a reference point of the camera. The origin of the vehicle coordinate system is located on the ground, and the height of the camera to be detected is defined by the z-coordinate of the reference point of the camera in the vehicle coordinate system. The roll, pitch and yaw angle to be determined denotes the angle of rotation of the image sensor about the x-axis, the y-axis and the z-axis of the vehicle coordinate system. The camera 13 is connected to determine its position and orientation with a calibration unit,--, and, ---- Once the position and orientation of the camera located in the vehicle has been determined, the height, roll, pitch and yaw angles of the camera relative to the vehicle coordinate system may be passed as parameters to an image evaluation algorithm of a driver assistance system…. The calibration unit is further configured to perform an optimization of the imaging model describing the imaging characteristics of the camera. By optimizing the imaging model, the height, roll angle and pitch angle of the camera are determined by means of the location of characteristic pixels in an image captured by the camera. --; in pages 3-4 of English version of EP 2166510 A1 as provided as NPL within this Office Action).
Re Claim 7, HANAWA as modified by XU, HOLD, YU and Sicconi and Zhang further disclose detecting the plurality of contours of the point of reference after detecting the edges of the image includes using a FINDCONTOURS function of an OPENCV2 library (see Sicconi: e.g., --further detection may be performed with the perspective shift. Lane detection methods may include, without limitation, color segmentation to pick out yellow and white lanes. Alternatively or additionally, a Canny threshold of the image may be taken, producing a detailed edge map of the scene. Canny edge map may capture more detail than necessary--, in [0114], and, --at step 1715 the motion detection analyzer 1324 determines a screen location on the digital screen 1308 of the rapid parameter change. In an embodiment, determining screen location may include identifying changing pixels according to a coordinate system as described above; identification may include, without limitation identifying coordinates of a boundary, geometric center, or the like of changed pixel area. Alternatively or additionally, determining screen location may include dividing the digital screen 1308 into a plurality of sections, regions of interest, and/or cells, for instance as described above, and identifying at least a cell and/or region of interest containing the rapid parameter change. A cell “contains” a rapid parameter change as used herein where the cell covers a portion of the digital screen 1308 where the rapid parameter change is occurring; in other words, a cell may “contain” the rapid parameter change where the cell contains at least a pixel undergoing the change. Identification may further include identifying a cell having a majority of changing pixels, a cell at a boundary and/or geometric center of a plurality of changing pixels, or the like.--, in [0128], and, -- Directional alerts may be generated to user based on detections of parameter changes and/or collision detection as described above. Collision detection routing may further isolate objects of motion in the image; these objects may then be passed through a classifier to be evaluated. Referring now to FIG. 23, a threshold method may classify objects from a heat map as described above. A threshold value is predefined, such as 165. A margin of error may be added. Further processing may be performed on the binarized blobs to filter certain aspect ratios and sizes. Bounding boxes of blobs may be generated and examined in the original image using a Haar Cascade, or any other suitable classifier. System may run on each individual frame using multi-threading to significantly improve response time. --, in [0142] {herein “segmentation” , and “identifying coordinates of a boundary, geometric center, or the like of changed pixel area.”, and/or “processing may be performed on the binarized blobs to filter certain aspect ratios and sizes. Bounding boxes of blobs may be generated and examined in the original image” read on claimed “contours”};
also see ZHANG: e.g., -- 1. Sobel operator extracting edge of image
after obtaining the picture of the driving in the road by the camera, firstly, the photo shot by the camera is
grey
wherein, respectively represent the red, green and blue RGB colour value in each pixel.
then extracting the vertical edge by Sobel operator. Order represents the grey information of an image,
wherein x and y represent the horizontal and vertical coordinate of the image pixel, the image is the partial
derivative of the horizontal direction and the vertical direction, and a pixel point of the image is The partial derivative in the angle direction can be respectively represented--, in page 8/14 of English version of ZHANG (CN 115713736 A), as provided as NPL with the Office Action; and,
-- if the angle equal to zero, namely representing the image has a longitudinal edge; otherwise, the image has a transverse edge; The edge information is extracted by edge density analysis based on line scanning.
1.2: Selected license plate
after performing binarization processing to the image, using cv2.findConfig in OpenCV, imutils.jin_contours function to search the contour of the detected object. We set a counter contours to count any object with a closed surface (below the python code):
searching the outline. Three input parameters: input image (binary image, black as background, white as target), contour search mode, contour approximation method. opencv2 returns two values: contours. The opencv3 returns to three values: img (image), countdown (contour), hierarchy (hierarchical structure)
Contours1=cv2.findContours(edged.copy(),cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE) – [10]; in pages 9-10/14 of English version of ZHANG (CN 115713736 A), as provided as NPL with the Office Action;
Also see: -- 2.1. The camera coordinate system of the camera
In order to systedescribe the coordinates of the object and image in the camera system, we define four coordinate systems: world coordinate system, camera coordinate system, image coordinate system and pixel coordinate system, the four coordinate systems are respectively appeared in each step of the camera imaging, the conversion relation between two two represents the conversion process in the imaging.
(1) world coordinate system: the three-dimensional coordinate system in the real world, for describing the specific position of any target in the reality physical space the origin of the coordinate system and the coordinate axis can be randomly selected, generally using represents;
(2) Camera coordinate system: observing the three-dimensional coordinate system of the reality physical space the angle of the camera, the origin of the coordinate system is selected as the imaging centre of the camera, the direction of the camera optical axis is the Z-axis of the coordinate system, generally using represents;
(3) Image coordinate system: defining a two-dimensional coordinate system on the imaging plane of the camera, for describing the target after camera imaging, the origin of the coordinate system is selected at the center of the imaging plane, generally using represents;--, in page 10/14 of English version of ZHANG (CN 115713736 A), as provided as NPL with the Office Action);
HANAWA (as modified by XU, HOLD and YU and Sicconi) and Zhang are combinable as they are in the same field of endeavor: processing images captured by vehicle camera . Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further modify HANAWA (as modified by XU, YU, and HOLD and Sicconi)’s method using Zhang’s teachings by including (1) "detecting a plurality of edges of the image" and, as a separate subsequent step, (2) "detecting a plurality of contours of the point of reference after detecting the edges of the image.", and even detecting a plurality of contours based on "the FINDCONTOURS function of the OPENCV2 library." to HANAWA (as modified by XU, YU and HOLD and Sicconi)’s image processing and performing contour detection and segmentations in order to search the contour of the detected object (see Zhang: e.g., in pages 8-10/14 of English version of ZHANG (CN 115713736 A), as provided as NPL with the Office Action).
Re Claim 8, HANAWA as modified by XU, HOLD, YU and Sicconi and ZHANG further disclose wherein the point of reference is a white rectangular plate (see YU: e.g., -- step 202, according to the position coordinate of the imaging point of each corner point in the image, the first position calibration plate is respectively connected with the target reference point of the vehicle, the position relation of the ground and the second position calibration plate are respectively connected with the target reference point of the vehicle, the position relation of the ground, determining the external reference of the camera.
In the implementation, the terminal device obtains the position coordinate of the imaging point of each corner point in the image, can be obtained at the first position, the position relation of the target reference point of the calibration plate and the vehicle, and obtaining the first position, the position relation of the calibration plate and the ground, and can be obtained in the second position. the position relationship between the calibration plate and the target reference point of the vehicle; and obtaining the position relationship between the calibration plate and the ground when obtaining the position relation between the calibration plate and the ground; then obtaining the position coordinate and the obtained position relation; determining the external parameter of the camera.--, in 1st-2nd para. of page 11 of English version of CN 111508027 A as provided as NPL within this Office Action).
Re Claim 9, HANAWA as modified by XU, HOLD, YU and Sicconi and ZHANG further disclose finding a center of the white rectangular plate in the image (see HANAWA: e.g., Fig. 3, and, -- The target area setting unit 21 sets the target area 300 at the position of such an infinite point, and causes the storage unit 203 to store an image of a distant landscape in the target area 300 (hereinafter referred to as a target image). As shown in FIG. 3, the target area 300 is a rectangular area of a predetermined size that is composed of a plurality of pixel values with the infinity point as the center. {herein “target area 300” aligns with claimed limitation of “a point of reference”}--, in pages 4-5 of English version of JP-2018148520-A as provided as NPL within this Office Action; and, --Here, a calculation method of the pitch angle of the camera 11 will be described with reference to FIGS. 9 and 10. As shown in FIG. 9, the coordinates on the captured image are set with the I coordinate in the right direction and the J coordinate in the upward direction with the lower left corner as the origin, and the unit is a pixel (for example, the size of the image in FIG. 9 is It is 752 pixels wide and 480 pixels long). Further, as shown in FIG. 10, the coordinates in the real space are the X axis in the horizontal direction, the Y axis in the height direction, and the Z axis in the straight traveling direction of the agricultural vehicle 1 with the position on the ground directly below the camera 11 as the origin. The axis shall be set.
The pitch angle of the camera 11 is θvc shown in FIG. 10A, and is the angle formed by the optical axis direction of the camera 11, that is, the screen center direction Jc, and the direction parallel to the ground, that is, the horizon direction Jv. . On the captured image as shown in FIG. 9, Jv is the J coordinate value in the horizon direction, Jc is the J coordinate of the screen center, and the pitch angle θvc is calculated by the following equation (P1).
tan (θvc) = PWH × (Jv−Jc) (P1)
Note that PWH is a tan value of the viewing angle per pixel.--, in pages 7-8 of English version of JP-2018148520-A as provided as NPL within this Office Action; also see HOLD: e.g., -- the position of the characteristic pixels in the camera image is determined by means of a modified radial symmetry transformation. The characteristic pixels include in the camera image usually a range of several pixels, if the resolution of the camera is large enough. By means of the modified radial symmetry transformation, for example, the centers of circular points in the camera image can be determined with subpixel accuracy… In order to completely determine the position and the orientation of the camera arranged in the vehicle, a yaw angle of the camera with respect to the vehicle coordinate system is preferably additionally determined.--, in pages 3-4 of English version of EP 2166510 A1 as provided as NPL within this Office Action).
Re Claim 10, HANAWA as modified by XU, HOLD, YU and Sicconi and Zhang further disclose determining the pixel position corresponding to the center of the white rectangular plate in the image (see HANAWA: e.g., Fig. 3, and, -- The target area setting unit 21 sets the target area 300 at the position of such an infinite point, and causes the storage unit 203 to store an image of a distant landscape in the target area 300 (hereinafter referred to as a target image). As shown in FIG. 3, the target area 300 is a rectangular area of a predetermined size that is composed of a plurality of pixel values with the infinity point as the center. {herein “target area 300” aligns with claimed limitation of “a point of reference”}--, in pages 4-5 of English version of JP-2018148520-A as provided as NPL within this Office Action; and, --Here, a calculation method of the pitch angle of the camera 11 will be described with reference to FIGS. 9 and 10. As shown in FIG. 9, the coordinates on the captured image are set with the I coordinate in the right direction and the J coordinate in the upward direction with the lower left corner as the origin, and the unit is a pixel (for example, the size of the image in FIG. 9 is It is 752 pixels wide and 480 pixels long). Further, as shown in FIG. 10, the coordinates in the real space are the X axis in the horizontal direction, the Y axis in the height direction, and the Z axis in the straight traveling direction of the agricultural vehicle 1 with the position on the ground directly below the camera 11 as the origin. The axis shall be set.
The pitch angle of the camera 11 is θvc shown in FIG. 10A, and is the angle formed by the optical axis direction of the camera 11, that is, the screen center direction Jc, and the direction parallel to the ground, that is, the horizon direction Jv. . On the captured image as shown in FIG. 9, Jv is the J coordinate value in the horizon direction, Jc is the J coordinate of the screen center, and the pitch angle θvc is calculated by the following equation (P1).
tan (θvc) = PWH × (Jv−Jc) (P1)
Note that PWH is a tan value of the viewing angle per pixel.--, in pages 7-8 of English version of JP-2018148520-A as provided as NPL within this Office Action; also see HOLD: e.g., -- the position of the characteristic pixels in the camera image is determined by means of a modified radial symmetry transformation. The characteristic pixels include in the camera image usually a range of several pixels, if the resolution of the camera is large enough. By means of the modified radial symmetry transformation, for example, the centers of circular points in the camera image can be determined with subpixel accuracy… In order to completely determine the position and the orientation of the camera arranged in the vehicle, a yaw angle of the camera with respect to the vehicle coordinate system is preferably additionally determined.--, in pages 3-4 of English version of EP 2166510 A1 as provided as NPL within this Office Action).
Re Claim 11, HANAWA as modified by XU, HOLD, YU and Sicconi and ZHANG further disclose transforming the pixel position corresponding to the center of the white rectangular plate into the determined pitch angle of the camera of the vehicle (see HANAWA: e.g., Fig. 3, and, -- The target area setting unit 21 sets the target area 300 at the position of such an infinite point, and causes the storage unit 203 to store an image of a distant landscape in the target area 300 (hereinafter referred to as a target image). As shown in FIG. 3, the target area 300 is a rectangular area of a predetermined size that is composed of a plurality of pixel values with the infinity point as the center. {herein “target area 300” aligns with claimed limitation of “a point of reference”}--, in pages 4-5 of English version of JP-2018148520-A as provided as NPL within this Office Action; and, --Here, a calculation method of the pitch angle of the camera 11 will be described with reference to FIGS. 9 and 10. As shown in FIG. 9, the coordinates on the captured image are set with the I coordinate in the right direction and the J coordinate in the upward direction with the lower left corner as the origin, and the unit is a pixel (for example, the size of the image in FIG. 9 is It is 752 pixels wide and 480 pixels long). Further, as shown in FIG. 10, the coordinates in the real space are the X axis in the horizontal direction, the Y axis in the height direction, and the Z axis in the straight traveling direction of the agricultural vehicle 1 with the position on the ground directly below the camera 11 as the origin. The axis shall be set.
The pitch angle of the camera 11 is θvc shown in FIG. 10A, and is the angle formed by the optical axis direction of the camera 11, that is, the screen center direction Jc, and the direction parallel to the ground, that is, the horizon direction Jv. . On the captured image as shown in FIG. 9, Jv is the J coordinate value in the horizon direction, Jc is the J coordinate of the screen center, and the pitch angle θvc is calculated by the following equation (P1).
tan (θvc) = PWH × (Jv−Jc) (P1)
Note that PWH is a tan value of the viewing angle per pixel.--, in pages 7-8 of English version of JP-2018148520-A as provided as NPL within this Office Action; also see HOLD: e.g., -- the position of the characteristic pixels in the camera image is determined by means of a modified radial symmetry transformation. The characteristic pixels include in the camera image usually a range of several pixels, if the resolution of the camera is large enough. By means of the modified radial symmetry transformation, for example, the centers of circular points in the camera image can be determined with subpixel accuracy… In order to completely determine the position and the orientation of the camera arranged in the vehicle, a yaw angle of the camera with respect to the vehicle coordinate system is preferably additionally determined.--, in pages 3-4 of English version of EP 2166510 A1 as provided as NPL within this Office Action).
Re Claim 12, claim 12 is the corresponding system claim to claim 1 respectively. Thus, claim 12 is rejected for the similar reasons as for claim 1. Furthermore, HANAWA as modified by XU and HOLD and YU and Sicconi and Zhang further disclose system for determining a pitch angle, comprising: a camera configured to capture an image, wherein the camera is coupled to a vehicle body of a vehicle; and a controller in communication with the camera, wherein the controller includes a processor and a non-transitory computer readable media in communication with the processor, and the controller is programmed to to perform operations (see HANAWA: e.g., --and an attachment parameter calculation part 26 that calculates a pitch angle and a yaw angle of a camera 11 from the position in the imaging image of the elimination point. The elimination point is calculated in the imaging image from a moving locus of the tracking regions set in the imaging image, and calculates the pitch angle and the yaw angle of the camera 11 from the elimination point in the imaging image. Thus, the attachment direction parameter calculation device can avoids that the attachment parameter is calculate--, in abstract, and, Fig. 12, Fig. 13, and, in: -- as coordinates in the image space, the upper left is defined as the origin, the right direction is defined as the X axis, and the lower direction is defined as the Y axis, and the angle between the straight line parallel to the X axis and the vanishing line is defined as the camera roll angle (camera (Rotation angle with the optical axis as the axis). Also, using the focal length of the camera, the coordinates of the vanishing point, and the roll angle calculated above, the pitch angle of the camera (rotation angle about the direction parallel to the moving plane and perpendicular to the optical axis of the camera) is calculated. calculate. Further, the camera yaw angle (rotation angle about the direction perpendicular to the moving plane) is calculated using the focal length of the camera, the coordinates of the deep vanishing point, the vanishing line, and the calculated pitch angle.--, in pages 2-3 of English version of JP-2018148520-A as provided as NPL within this Office Action).
Re Claims 19-20, claims 19-20 are the corresponding method claim to claims 1, 8, and 10-11 respectively. Thus, claims 19-20 are rejected for the similar reasons as for claims 1, 8, and 10-11. Furthermore, HANAWA as modified by XU, HOLD. YU and Sicconi and ZHANG further disclose a method for determining a pitch angle of a camera in a vehicle (see HANAWA: e.g., --and an attachment parameter calculation part 26 that calculates a pitch angle and a yaw angle of a camera 11 from the position in the imaging image of the elimination point. The elimination point is calculated in the imaging image from a moving locus of the tracking regions set in the imaging image, and calculates the pitch angle and the yaw angle of the camera 11 from the elimination point in the imaging image. Thus, the attachment direction parameter calculation device can avoids that the attachment parameter is calculate--, in abstract, and, Fig. 12, Fig. 13, and, in: -- as coordinates in the image space, the upper left is defined as the origin, the right direction is defined as the X axis, and the lower direction is defined as the Y axis, and the angle between the straight line parallel to the X axis and the vanishing line is defined as the camera roll angle (camera (Rotation angle with the optical axis as the axis). Also, using the focal length of the camera, the coordinates of the vanishing point, and the roll angle calculated above, the pitch angle of the camera (rotation angle about the direction parallel to the moving plane and perpendicular to the optical axis of the camera) is calculated. calculate. Further, the camera yaw angle (rotation angle about the direction perpendicular to the moving plane) is calculated using the focal length of the camera, the coordinates of the deep vanishing point, the vanishing line, and the calculated pitch angle.--, in pages 2-3 of English version of JP-2018148520-A as provided as NPL within this Office Action).
Conclusion
Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WEI WEN YANG whose telephone number is (571)270-5670. The examiner can normally be reached on 8:00 - 5:00 pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached on 571-272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WEI WEN YANG/Primary Examiner, Art Unit 2662
Prosecution Insights