DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-2, 5-6, and 15-22 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Kikuchi et al. ( Machine translated Japanese Patent Publication JP 2022030436 A, “Kikuchi”).
Kikuchi teaches, 1. An image display device (Fig. 4) comprising:
a processor (Fig.4 CPU 201),
wherein the processor is configured to:
perform control to display, on a display, a display screen having a display region for a partial image that is a portion of a long image, ( Page 7 Lines 64-67, “The display control unit 23 is mainly realized by the processing of the CPU 201, and causes the display 206 to display various images. The display control unit 23 displays, for example, the image viewing screens 300, 400, 500, which will be described later, on the display 206”. The display region 350 displays a portion of the portion of a long image based on capture position 383.) which is obtained by imaging a structure extending in a first direction with an imaging device and in which a long side direction is along the first direction, and a first index bar which extends in the first direction and indicates a first distance range that is at least a portion of a distance range of the structure in the first direction and on which a capture position mark indicating a capture position of the partial image, which is being displayed in the display region, in the structure is displayed. ( Fig. 11 and 12, the first index bar is the position bar indicating start to end position of the tunnel and long side direction is horizontal. Element 383 is the first capture mark where is the image is captured from. The image corresponding to the capture position is shown on display area 350. The first index bar extends in the first direction which is horizonal. Page 14, Lines 27-40: “The inspection worker moves the slider-type magnification adjusting means 381 as shown in FIG. 11 by using the pointer p1 using the input means, so that the captured image displayed in the active display area is displayed. Enlarge or reduce. The reception unit 22 receives an operation of enlarging or reducing the captured image displayed in the active display area by the operation of the inspection worker with respect to the magnification adjusting means 381. The inspection worker performs a movement operation of the slider type shooting position selection means 383 as shown in FIG. 11 with the pointer p1 using the input means to enlarge the image to be displayed in the enlarged image display area 350.”)
Claim 21 is directed to a method and its steps are similar in scope and function of the element of device claim 1 and therefore claim 21 is rejected with same rationales as specified in the rejection of claim 1.
Claim 22 is directed to a non-transitory computer readable medium (Column 7 Lines 71-74: “Of these, the CPU 911 controls the operation of the entire image pickup unit 900. The CPU 911 reads a program, data, setting information, and the like from the ROM 912 and the like onto the RAM 913, and executes the process”) and its elements are similar in scope and function of the element of device claim 1 and therefore claim 22 is rejected with same rationales as specified in the rejection of claim 1.
Regarding claim 2, Kikuchi teaches, wherein the processor is configured to: receive a designation of any position of the first index bar; and display the partial image, whose capture position corresponds to the designated position, in the display region. (Refer to Fig.11 and 12, element 350 displays the image captured from position 383. Page 14, Lines 27-40 “The inspection worker moves the slider-type magnification adjusting means 381 as shown in FIG. 11 by using the pointer p1 using the input means, so that the captured image displayed in the active display area is displayed. Enlarge or reduce. The reception unit 22 receives an operation of enlarging or reducing the captured image displayed in the active display area by the operation of the inspection worker with respect to the magnification adjusting means 381. The inspection worker performs a movement operation of the slider type shooting position selection means 383 as shown in FIG. 11 with the pointer p1 using the input means to enlarge the image to be displayed in the enlarged image display area 350. “)
Regarding claim 5, Kikuchi teaches, wherein the partial image is capable of being displayed to be enlarged in the display region, ( Page 16 Lines 14-23: “Next, the display control unit 23 displays the enlarged image corresponding to the abnormality confirmation position received in step S53, that is, the position of the selected flag image, in the enlarged image display area 350 (step S54). FIG. 12 shows a screen example of the image viewing screen 300 displayed in step S54. The image viewing screen 300 shown in FIG. 12 displays an enlarged image corresponding to the shooting position “6” of the bird's-eye view image displayed in the bird's-eye view image display area 310 in the enlarged image display area 350.”) and
an auxiliary window (Fig. 12 element 360) indicating which position of the partial image the portion displayed to be enlarged corresponds to is capable of being displayed. (Page 16 Lines 26-30 “In the case of the example shown in FIG. 12, the illuminated portion shown in the enlarged image is an abnormal image of overexposure, and the flag display area 360 corresponds to the illuminated portion shown in the enlarged image”).
Regarding claim 6, Kikuchi teaches, wherein an operation portion for inputting an instruction to switch the partial image being displayed in the display region to a partial image adjacent to the partial image being displayed in the display region is provided on the display screen. ( Page 14, Lines 37-40: “Switch images. The reception unit 22 receives the selection of the shooting position of the shot image to be displayed in the enlarged image display area 350 by the operation of the inspection worker with respect to the shooting position selection means 383.”)
Regarding claim 15, Kikuchi teaches, wherein the long image is an image obtained by combining a plurality of captured images obtained by imaging different portions of the structure with the imaging device a plurality of times along at least the first direction of the first direction and a second direction intersecting the first direction. ((Page 4 Lines 1-6: “The camera unit 901 is a line camera including a plurality of cameras and equipped with a line sensor in which photoelectric conversion elements are arranged in one row or a plurality of rows. The camera units 901 are installed so that their angles of view partially overlap with the adjacent cameras, and the entire camera unit is installed so that half of the tunnel in the circumferential direction fits in the angle of view. “
Page 12, lines 67-74: “Of these, the bird's-eye view image display area 310 displays a bird's-eye view image showing the entire tunnel 6. The bird's-eye view image of the tunnel 6 is an image in which a plurality of captured images are arranged in the shooting order so that the entire tunnel 6 (from the entrance to the exit) can be viewed from the bird's-eye view. The horizontal direction of the bird's-eye view image is the traveling direction of the photographing vehicle 9, and the vertical direction is the circumferential direction of the tunnel 6.”).
Regarding claim 16, Kikuchi teaches, wherein the plurality of captured images are images obtained by imaging the structure at a plurality of different positions in the first direction while moving the imaging device in the first direction. (Page 12, lines 67-74: “Of these, the bird's-eye view image display area 310 displays a bird's-eye view image showing the entire tunnel 6. The bird's-eye view image of the tunnel 6 is an image in which a plurality of captured images are arranged in the shooting order so that the entire tunnel 6 (from the entrance to the exit) can be viewed from the bird's-eye view. The horizontal direction of the bird's-eye view image is the traveling direction of the photographing vehicle 9, and the vertical direction is the circumferential direction of the tunnel 6.”)
Regarding claim 17, Kikuchi teaches, wherein the imaging device has a plurality of cameras that are arranged along the second direction, and the plurality of captured images are images obtained by imaging the structure with the plurality of cameras. (Page 12, lines 67-74: “Of these, the bird's-eye view image display area 310 displays a bird's-eye view image showing the entire tunnel 6. The bird's-eye view image of the tunnel 6 is an image in which a plurality of captured images are arranged in the shooting order so that the entire tunnel 6 (from the entrance to the exit) can be viewed from the bird's-eye view. The horizontal direction of the bird's-eye view image is the traveling direction of the photographing vehicle 9, and the vertical direction is the circumferential direction of the tunnel 6.”)
Regarding claim 18, Kikuchi teaches, wherein the imaging device has a plurality of cameras that are arranged along the first direction, and the plurality of captured images are images obtained by imaging the structure with the plurality of cameras. (Page 12, lines 67-74: “Of these, the bird's-eye view image display area 310 displays a bird's-eye view image showing the entire tunnel 6. The bird's-eye view image of the tunnel 6 is an image in which a plurality of captured images are arranged in the shooting order so that the entire tunnel 6 (from the entrance to the exit) can be viewed from the bird's-eye view. The horizontal direction of the bird's-eye view image is the traveling direction of the photographing vehicle 9, and the vertical direction is the circumferential direction of the tunnel 6.”)
Regarding claim 19, Kikuchi teaches, wherein the imaging device has a plurality of cameras that are arranged along the second direction, the plurality of captured images are images obtained by imaging the structure with the plurality of cameras while moving the imaging device along the first direction, (Page 4 Lines 106: “The camera unit 901 is a line camera including a plurality of cameras and equipped with a line sensor in which photoelectric conversion elements are arranged in one row or a plurality of rows. The camera units 901 are installed so that their angles of view partially overlap with the adjacent cameras, and the entire camera unit is installed so that half of the tunnel in the circumferential direction fits in the angle of view. “) and
the long image is an image obtained by combining the plurality of captured images along the first direction and the second direction. (Page 12, lines 67-74: “Of these, the bird's-eye view image display area 310 displays a bird's-eye view image showing the entire tunnel 6. The bird's-eye view image of the tunnel 6 is an image in which a plurality of captured images are arranged in the shooting order so that the entire tunnel 6 (from the entrance to the exit) can be viewed from the bird's-eye view. The horizontal direction of the bird's-eye view image is the traveling direction of the photographing vehicle 9, and the vertical direction is the circumferential direction of the tunnel 6.”)
Regarding claim 20, Kikuchi teaches, wherein the structure is a tunnel, the first direction is a length direction connecting an entrance and an exit of the tunnel, and the second direction is a circumferential direction of the tunnel. (Page 12, lines 67-74: “Of these, the bird's-eye view image display area 310 displays a bird's-eye view image showing the entire tunnel 6. The bird's-eye view image of the tunnel 6 is an image in which a plurality of captured images are arranged in the shooting order so that the entire tunnel 6 (from the entrance to the exit) can be viewed from the bird's-eye view. The horizontal direction of the bird's-eye view image is the traveling direction of the photographing vehicle 9, and the vertical direction is the circumferential direction of the tunnel 6.”)
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 3-4 are rejected under 35 U.S.C. 103 as being unpatentable over Kikuchi in view of Bhargava et al. (US patent publication: 20180349005, “Bhargava”).
Regarding claim 3, Kikuchi doesn’t expressly teach, wherein a second index bar that indicates a second distance range wider than the first distance range and indicates which position in the second distance range the first index bar being displayed corresponds to is capable of being displayed.
Bhargava teaches, wherein a second index bar that indicates a second distance range wider than the first distance range and indicates which position in the second distance range the first index bar being displayed corresponds to is capable of being displayed. (Refer to Fig. 1, slider 122 with first precision level is the second index bar, slider 122 with second precision level is the first index bar, second index bar is wider than the first index bar. “[0027] Responsive to a detection of one of these triggering interactions, the computing device alters the precision level of the slider. To do so, the computing device computes a range of values to be used for a display of the slider at a second precision level, e.g., based on an available display area of the display device. At the second precision level, a display range of the slider includes less than the full range of potential values for the corresponding parameter. For example, instead of providing 100 different values along a three-inch slider at the first precision level, the three-inch slider can provide 20 different values at the second precision level to accommodate a display size of the display device.”)
Kikuchi and Bhargava are analogous as they are from the field of display processing of object in user interface.
Therefore it would have been obvious for an ordinary skilled art person in the art before the effective filing date of the claimed invention to have modified Kikuchi to have included a second index bar that indicates a second distance range wider than the first distance range and indicates which position in the second distance range the first index bar being displayed corresponds to is capable of being displayed as taught by Bhargava.
The motivation to include the modification is to provide better controllability to select a location from index bar and see the relationship of selected location corresponding to full length of tunnel.
Regarding claim 4, Kikuchi as modified by Bhargava teaches, wherein the second distance range is an entire distance range of the structure in the first direction. (Bhargava Fig. 1 slider 122 with first precision level is the claimed second index bar which has full distance range)
Claim(s) 7-10 are rejected under 35 U.S.C. 103 as being unpatentable over Kikuchi.
Regarding claim 7, Kikuchi teaches, wherein a portion-of-interest mark indicating a portion of interest in the long image is capable of being displayed (Fig 11 And 12, flag display area 320 shows different marks representing different points of interest in the long image. Page 13 column 14-32: “The flag display area 320 displays the flag image at a position where the bird's-eye view image displayed in the bird's-eye view image display area 310 and the traveling direction of the shooting vehicle 9 are synchronized. The display control unit 23 reads the flag information written in the footer area of the captured image, and displays the flag image synchronized with the bird's-eye view image in the traveling direction. The inspection worker causes an abnormality by checking the flag image displayed at the same position in the traveling direction (horizontal direction) as the place where the abnormality is detected in the bird's-eye view image displayed in the bird's-eye view image display area 310. You can grasp the location where you are. Further, in the flag display area 320, for example, when there are a plurality of detection results, the display method such as the color or shape of the flag image may be changed, or a plurality of flag images may be vertically arranged and displayed. In the example of FIG. 11, the flag display area 320 is the location and the traveling direction (horizontal direction) of the overexposed area (for example, the lighting in the tunnel 6) in the bird's-eye view image displayed in the bird's-eye view image display area 310. A black flag image is displayed at the same position.”) but doesn’t teach so on the first index bar.
Kikuchi has a first index bar (Fig. 11 bar containing element 383) and a flag display area 320 containing portion-of-interest marks.
It would have been obvious for an ordinary skilled person in the art before the effective fling date of the claimed invention to have modified Kikuchi by including flags or portion-of-interest marks on first index bar for having a compact display of both information (pointer 383 to move capture position of and flag/portion-of-interest marks) on the same slider.
The motivation for the modification is to provide a single bar that can be used to set capture position to view the point-of-interest easily.
Regarding claim 8, Kikuchi as modified teaches, wherein the portion of interest includes at least one of a portion designated by a user or a portion in which a specific object is present. (Kikuchi, Page 13 column 21-24: “The inspection worker causes an abnormality by checking the flag image displayed at the same position in the traveling direction (horizontal direction) as the place where the abnormality is detected in the bird's-eye view image displayed in the bird's-eye view image display area 310.” Portion of interest shows a position n in which a specific object(where abnormality is present) is present.)
Regarding claim 9, Kikuchi as modified teaches, wherein a text memo or a voice memo input by the user is displayed in association with the portion of interest. (Page 2, Lines 57-71: “In FIG. 1, the inspection worker who is inspecting on the inspection vehicle 7 overwrites the deformed portion such as a crack with a special choke to make the deformed portion conspicuous, or the width of the crack is several centimeters. Or etc. At that time, the inspection worker writes a comment showing the state of the deformation and the evaluation result in the field note or the like. On the other hand, the assistant under the inspection vehicle 7 writes the comments made by the inspection worker in a field note or the like, and in some cases, takes a picture of the whole. A small microphone and a small camera may be provided on the helmet worn by the inspection worker, comments made by the inspection worker may be recorded by the small microphone, and the commented target portion may be photographed by the camera. In this case, the recorded voice information may be recognized by a voice recognition device, digitized, and automatically recorded in a field note (in this case, a tablet terminal or the like) together with an image taken by a small camera.” Inspection worker’s voice information on crack is recording in a field note.
It would have been obvious for an ordinary skilled person in the art before the effective filing date of the claimed invention to have modified Kikuchi as modified to have a text memo or a voice memo input by the user is displayed in association with the portion of interest for the purpose of providing user details of failure/issue along with displayed portion of capture image).
Regarding claim 10, Kikuchi as modified teaches, wherein the portion designated by the user is at least one of a portion designated by an imaging person of the long image during capture of the long image or a portion designated by an inspector of the long image during inspection of the long image.(Page 13 column 21-24: “The inspection worker causes an abnormality by checking the flag image displayed at the same position in the traveling direction (horizontal direction) as the place where the abnormality is detected in the bird's-eye view image displayed in the bird's-eye view image display area 310.” Portion of interest shows a position n in which a specific object(where abnormality is present) is present and in designated by inspector.)
Claim(s) 11-13 are rejected under 35 U.S.C. 103 as being unpatentable over Kikuchi in view of Motominami ( US 20070258655, “ Motominami”).
Regarding claim 11, Kikuchi doesn’t expressly teach, wherein a portion whose partial image is displayed and a portion whose partial image is not displayed are capable of being displayed to be distinguished from each other on the first index bar.
However, Motominami teaches, wherein a portion whose partial image is displayed and a portion whose partial image is not displayed are capable of being displayed to be distinguished from each other on a first index bar. ( Refer to Fig. 8, 62b is the range for which the image is displayed. The images corresponding to area outside of the 62b range is not displayed. 62b and other area on the bar are clearly distinguishable “[0072] That is, when the right key of the cross key 21 is pressed, the single frame selector 53 continuously moves the display position of the pointer image 62a in the slide bar image 62 to the end point of the motion image during pressing, sets a display position of the pointer image 62a at a press completion time point to the designated point, extracts the still image data of the designated point, and displays the still image”)
Kikuchi and Motominami are analogous as they are from the display processing of images.
Therefore it would have been obvious for an ordinary skilled person in the art before the effective filing date of the claimed invention to have modified Kikuchi to have included wherein a portion whose partial image is displayed and a portion whose partial image is not displayed are capable of being displayed to be distinguished from each other on a first index bar as taught by Motominami.
The motivation to include the modification is to provide use idea of position range of currently displayed partial image.
Regarding claim 12, Kikuchi as modified by Motominami teaches, wherein a display completion mark indicating the portion whose partial image is displayed is displayed on the first index bar such that the portion whose partial image is displayed and the portion whose partial image is not displayed are displayed to be distinguished from each other. (Motominami Fig, 8, the index bar has range 62b which has a completion mark 62a and based on the completion mark on the bar, he portion whose partial image is displayed and the portion whose partial image is not displayed are displayed to be distinguished from each other.”)
Regarding claim 13, Kikuchi as modified by Motominami teaches, wherein at least one of a brightness, color, or pattern of the first index bar is changed such that the portion whose partial image is displayed and the portion whose partial image is not displayed are displayed to be distinguished from each other. (Motominami Fig, 8, the index bar has range 62b on the bar, 62b has different pattern than the other area on the bar. The image of the other area on the bar is not displayed. ”)
Claim(s) 14 is rejected under 35 U.S.C. 103 as being unpatentable over Kikuchi in view of Cho et al. ( US Patent Publication: 20190220167, “Cho”).
Kikuchi doesn’t expressly teach, wherein the processor is configured to: receive a designation of an object to be measured in the partial image; and display at least one of a position of the object to be measured in the structure or an actual size of the object to be measured in response to the designation.
However, Cho teaches, receive a designation of an object to be measured in an image; and display at least one of a position of the object to be measured in a structure or an actual size of the object to be measured in response to the designation. ( Fig. 21 displays an image and user selects a target object to be measured on the object. Fig. 23 displays display at least one of a position of the object to be measured in a structure. “[0207] To solve the problem described above, an image measuring method according to the fifth embodiment includes the steps of displaying an image of a measurement target object on a touch panel display, accepting touch input operation of specifying a position in the image displayed on the touch panel display, and searching for and detecting an edge present in a predetermined range around the specified position specified in the touch operation accepting step. The image measuring method is characterized in that when a plurality of edges are found in the vicinity of the specified position, a control object for selecting each of the edges is displayed on the touch panel display not only in an aspect that prevents a wrong edge from being selected through the touch operation but in an aspect that allows visual recognition of the correspondence between each of the edges and the corresponding control object.”)
Kikuchi and Cho are analogous as they are from the field of display processing of object in user interface.
Therefore it would have been obvious for an ordinary skilled art person in the art before the effective filing date of the claimed invention to have modified Kikuchi to have included to receive a designation of an object to be measured in an image; and display at least one of a position of the object to be measured in a structure or an actual size of the object to be measured in response to the designation similar to receiving a designation of an object to be measured in an image; and display at least one of a position of the object to be measured in a structure or an actual size of the object to be measured in response to the designation as taught by Cho.
The motivation to include the modification is to provide the user to visually display the details of a defect in the structure.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Sato et al. ( US patent Publication 2018/0218534) Para [100] teaches displaying image of a structure and correction of shape..
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SAPTARSHI MAZUMDER whose telephone number is (571)270-3454. The examiner can normally be reached 8 am-4 pm PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Said Broome can be reached at (571)272-2931. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SAPTARSHI MAZUMDER/ Primary Examiner, Art Unit 2612
Prosecution Insights