Detailed Action
1. Claims 1-13 are pending in this Application.
Notice of Pre-AIA or AIA Status
2. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
3. Claim 13 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter as follows. Claim13 as a whole define “ A program that causes a computer system to execute the steps …..”, which is considered as program per se. The specification in paragraphs [0013]-[0104] states “Specifically, the information processing method according to the present technology is realized by loading a program that configures the software, which has been stored in the ROM 102 or the like, to the RAM 103 and executing it. The information processing apparatus 20 installs the program via the recording medium 111, for example. Alternatively, the information processing apparatus 20 may install the program via a global network or the like. Otherwise, any computer-readable non-transitory storage medium may be used”. This statement of the specification may include transitory and non-transitory propagation signals, and “a transitory, propagating signal … is not a “process, machine, manufacture, or composition of matter.” Those four categories define the explicit scope and reach of subject matter patentable under 35 U.S.C. § 101; thus, such a signal cannot be patentable subject matter.” Therefore, the examiner suggests: Amending the claim to embody the program on “non- transitory computer-readable storage medium” or equivalent that excludes computer readable medium as a “signal”, “carrier wave”, or “transmission medium” which are deemed non-statutory.
Any amendment to the claim should be commensurate with its corresponding disclosure
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103, which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
4. Claims 1,3-4 and 11-13 are rejected under 35 U.S.C. 103(a) as being unpatentable over YOSHIMURA et al., (hereafter YOSHIMURA ), JP2016126188A, pub., 07/11/2016 ,in view of Dayal; Rajiv (hereafter Dayal ), US-20160210834A, pub. on 07/21/2016.
As to claim 1, YOSHIMURA teaches an information processing apparatus (Abstract, [0005], An audio information display device worn and used by a user,) , comprising:
an estimation unit that estimates a position of a sound source on a basis of an environmental sound and a captured image around a user(Claim 1, [0005], voice signal generation unit that collects sound at multiple locations and generates a voice signal; a sound source location identification unit that identifies the location of a sound source based on the voice signal generated by the voice signal generation unit; an image generation and display unit that generates and displays an image based on the voice signal generated by the voice signal generation unit; and determination unit that determines whether or not the location of the sound source identified by the sound source location identification unit is within the user's field of vision,) ;
a first generation unit that generates text information obtained by converting the environmental sound into text ( claim 5, [0008], a string generation unit that performs speech recognition processing on the audio signal generated by the audio signal generation unit to generate a string of corresponding to the human voice when the audio signal generated by the audio signal of characters corresponding to the specific audio signal component.
a calculation unit that calculates an attention score regarding a risk level of the user on a basis of the text information([0042]-[0043], the risk point correspondence table 800 is composed of items 810, conditions 820, and points 830 that are associated with each other. Item 810 is information indicating the items that are used to calculate the risk score. Points 830 are information indicating the points that will be added to the risk score if the corresponding condition 820 is met for the corresponding item 810.);
It is noted that Yoshimura fails to disclose the underlined portion of the limitation referenced below:
“recognition unit that recognizes object information regarding an object on a basis of the captured image; and a second generation unit that generates presentation expression information to be presented to the user on the object information”,
On the other hand in the same field of endeavor a wearable smart method and device for warning a user of a hazard based on image data and audio data of Dayal teaches a second generation unit that generates presentation expression information to be presented to the user on a basis of the position of the sound source, the text information, the attention score (Figs. 5-8, [0024], [0095]-[0120] ,FIG. 8 illustrates an exemplary database 800 that may be utilized by the processor 111 to determine an estimated distance to a hazard based on a detected volume of the hazard. The database 800 may include a first column 802 that corresponds to a particular hazard, a second column 804 that corresponds to various volumes of the hazard and a third column 806 that corresponds to an estimated distance to the hazard based on the detected volume).
It would have been obvious to a person of ordinary skill in the art before the effective
filing date of the claimed invention to incorporate a wearable smart system for providing hazard warning information based on image data and audio data taught by Dayal into Yoshimura
The motivation for doing so is to provide Yoshimura users with a highly robust and accurate hazard-detection system. The system assesses the severity and confidence level of each hazard, identifies dangerous unknown objects, and provides actionable context by pinpointing the distance and direction of the hazard relative to the user (see [0025]).
As to claim 3, YOSHIMURA teaches the text information includes at least one of a sound category, the position of the sound source, an onomatopoeia, or a sound volume (Claim 5, [0102], the display unit 1670 generates sound source position information indicating the position of the MFP 1500 (for example, an arrow pointing in the direction of the MFP 1500) (step S2035), and displays it together with the onomatopoeia and string generated by the image generation unit 1660 (step S2040), where a string generation unit that performs speech recognition processing on the audio signal generated by the audio signal generation unit to generate a string corresponding to the human).
As to claim 4, YOSHIMURA the object information includes at least one of a position, a size, a category, or a movement velocity of the object ([0055], method of determining the size of the source image from the volume calculated by the volume calculation unit 210,
Once the size and color of the original image are determined, the image generation unit 260 refers to the movement direction scaling ratio correspondence table 600 to determine the scaling ratio 620 from the movement direction calculated by the movement determination unit 220);
As to claim 11, YOSHIMURA teaches the object the calculation unit dynamically changes the attention score preset to the text information on a basis of the captured image (Fig.11, [0063]-[0065], The system calculates risk level points based on captured image data, and evaluates whether they exceed a predetermined threshold. If the threshold is exceeded, a vibrator is activated to notify the user. If the score is below the threshold or the vibration sequence finishes, the device terminates its vibration processing. Thus, risk level points is dynamically change based on the type of captured image data)
.
Claim 12 is rejected the same as claim 1 except claim 12 is directed to a method claim. Thus, argument analogous to that presented above for claim 1 is applicable to claim 12.
As to claim 13, YOSHIMURA teaches a program that causes a computer system to execute the steps ([0096], Figure 19 shows an example of the display that the display unit 1670 As shown in the figure, the display unit 1670 displays the string of characters (display string 1900) generated by the image generation unit 1660 at a predetermined position. Each of the above functional blocks is implemented by the processor executing a program stored in memory); regarding remaining limitations of claim 13, all the remaining limitations are rejected the same as claims 1.
5. Claims 2, 5-7 and 9-10 are rejected under 35 U.S.C. 103(a) as being unpatentable over YOSHIMURA ), JP2016126188A,in view of Dayal, US-20160210834A, and further in view of Janugani et al., (hereafter Janugani),US 20200177722 A1, pub. 06/04/2020
As to claim 2, YOSHIMURA teaches the presentation expression information includes a text sentence to be presented to the user ([0094], As shown in Figures 17(b) and 18(b), the display unit 1670 displays the generated sound source location information (arrows 1770 and 1870) together with onomatopoeia (onomatopoeia 1760 and 1860) and strings (display strings 1750 and 1850) generated by the image generation unit 1660.),
It is noted that the combination of YOSHIMURA and Dayal does not specifically teaches “a size of the text sentence, a presentation position of the text sentence, and an expression effect applied to the text sentence”
On the other hand Janugani a size of the text sentence, a presentation position of the text sentence, and an expression effect applied to the text sentence([0031] A method for presenting notifications on a display screen of a presentation device 120, comprising the steps of determining a display duration, determining a display location on the screen, and rendering visual attributes of the notification including transparency, color, shape, and font size.)
It would have been obvious to a person of ordinary skill in the art before the effective
filing date of the claimed invention to incorporate a method of displaying warning notifications using a combination of transparency, color, shape, and typography taught by Janugani into modified Yoshimura.
The motivation for doing so allows use of modified Yoshimura to instantly recognize the urgency of a message, and ensures accessibility for users with visual or cognitive impairments
As to claim 5, Dayal teaches the second generation unit determines the presentation position of the text sentence on a basis of a behavior of the user([0100], The method also includes determining, by the processor, that the current location is within a predetermined distance of the location of the hazard. The method also includes determining, by the processor, feedback data to be provided to the user in response to the current location being within the predetermined distance of the location of the hazard, wherein the feedback data provided to the user if the user is approaching the location of the hazard).
It would have been obvious to a person of ordinary skill in the art before the effective
filing date of the claimed invention to incorporate a wearable smart system for providing hazard warning information based on image data and audio data taught by Dayal into Yoshimura
The motivation for doing so is to provide Yoshimura users with a highly robust and accurate hazard-detection system. The system assesses the severity and confidence level of each hazard, identifies dangerous unknown objects, and provides feedback to the user in response to the current location being within the predetermined distance of the location of the hazard (see [0025], [0100]).
As to claim 6, YOSHIMURA teaches the second generation unit presents the text sentence in a periphery of the position of the sound source in a case where the position of the sound source is inside an angle-of-view of an imaging device that acquires the captured image ([0066]-[0069] As shown in Figure 12(a), when the person speaking 1200 is within the field of view, the displayed string 1210, which is generated by the string generation unit 205 through speech recognition, is displayed near the person speaking 1200 with its size and brightness reduced.
PNG
media_image1.png
262
358
media_image1.png
Greyscale
PNG
media_image2.png
220
332
media_image2.png
Greyscale
As to claim 7, YOSHIMURA teaches the second generation unit presents the text sentence so as not to cover the object in a case where the position of the sound source is inside an angle-of-view of an imaging device that acquires the captured image(([0066]-[0069] as discussed in claim 6 above As shown in Figure 12(a), when the person speaking 1200 is within the field of view, the displayed string 1210, which is generated by the string generation unit 205 through speech recognition, is displayed near the person speaking 1200 with its size and brightness reduced. As shown in Fig.12 the text messages are located at the right corner of the voice information display device 100, thus, the text message does not cover the user view. )
As to claim 9, YOSHIMURA teaches the second generation unit determines ([0042], [0101], Item 810 is information indicating the items that are used to calculate the risk score, the warning sound and a string of characters indicating the meaning of the warning sound, which are stored in advance in association with that warning sound, where the risk score is associated to the warning sound);
However, it is noted that the modified YOSHIMURA does not specifically the “size of the text sentence”
On the other hand Janugani teaches size of the text sentence ([0031], method of displaying the notification on the presentation device 120, that includes where the notification should appear on the screen, how the notification looks((e.g., transparency, color, shape, and font size).
It would have been obvious to a person of ordinary skill in the art before the effective
filing date of the claimed invention to incorporate a method of displaying warning notifications using a combination of transparency, color, shape, and typography taught by Janugani into modified Yoshimura.
The motivation for doing so allows use of modified Yoshimura to instantly recognize the urgency of a message, and ensures accessibility for users with visual or cognitive impairments
As to claim 10, YOSHIMURA teaches the second generation unit presents the text sentence and the position of the sound source in a case where the position of the sound source is outside an angle-of-view of an imaging device that acquires the captured image ( Fig.12 [0068], On the other hand, as shown in Figure 13(b), if the ambulance 1350 is outside the field of view, the symbol 1370 indicating the ambulance siren(sound source) and the displayed text 1390 are displayed together with the arrow 1380 indicating the direction of the ambulance 1350, without any reduction in size or brightness.)
Allowable Subject Matter
6. Claim 8 is objected to as being dependent upon a rejected base claims, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
7. Regarding independent claim 8 no prior art is found to anticipate or render the flowing limitation obvious “wherein the object information includes a covering cost regarding a degree of importance of the object, and the second generation unit determines the presentation position of the text sentence on a basis of the covering cost”
Prior art not used in rejections but pertinent to the claims or disclosure.
“ Design and Development of Portable Navigation System for Disabled Person using Image, Text and Audio”, IEEE, 11-13 February 2022, to Akshata Supekar et al., disclosed:
Visually impaired person experience many issues related to communication, navigation, information access and movement. This impacts their living or economic condition, where the families also suffers and further it affects the society as well. Due to the disability, while walking they often get hit by objects like poles, stones, walls, cars, people etc. Usually, conventional methods are used to support their mobility which includes the long white cane known as smart stick, guide dogs which are the footpath's route finders. After using several methods of communication, they face many challenges while moving in unknown open places. As a result they may get injured severely, and sometimes they face humiliation and lose confidence. There are chances of getting lost also. It is extremely difficult for family members to locate them in such instances. Hence they always need to carry the stick, wherein the obstacles can be identified by tapping. This solution, however, is ineffective for them. A sophisticated smart stick or device is needed to avoid obstructions. Such device can be helpful to detect the object from a far distance, and they can be saved from mishaps. However if unexpected change of environment occurs, these conventional methods are not useful. Researchers are working towards improvement in navigational aids which can help such disabled people. The design of novel navigation system with cutting-edge technology, like wearable computer and ubiquitous computing design along with development of communication support technology and position identification technology are essential (see Abstract Figs. 1-3)
Contact Information
Any inquiry concerning this communication or earlier communication from the examiner should be directed to Mekonen Bekele whose telephone number is (469) 295-9077.The examiner can normally be reached on Monday-Friday from 9:00AM to 6:50 PM Eastern Time.
If attempt to reach the examiner by telephone are unsuccessful, the examiner’s supervisor Eng, George can be reached on (571) 272-7495.The fax phone number for the organization where the application or proceeding is assigned is 571-237-8300. Information regarding the status of an application may be obtained from the patent Application Information Retrieval (PAIR) system. Status information for published application may be obtained from either Private PAIR or Public PAIR. Status information for unpublished application is available through Privet PAIR only.
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have question on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866.
/MEKONEN T BEKELE/Primary Examiner, Art Unit 2699
Prosecution Insights